Performance

What the memory API costs in latency and tokens, measured rather than estimated, with the method behind each number.

Last updated 31 July 2026.

Latency

Wall-clock timings from a client calling the hosted API over the public internet. Six samples per operation against a memory with online web search enabled, so these include network transit, retrieval, and — for the generating operations — a full model round trip.

Operation	p50	p95	Tokens	Credits
analytics	90 ms	159 ms	—	0
sources	152 ms	304 ms	—	0
status	319 ms	364 ms	—	0
boolean_ask	398 ms	544 ms	—	0
search	972 ms	1.08 s	—	0
ask	3.93 s	6.40 s	~1,071	~0.11
chat	12.76 s	17.51 s	~2,387	~0.24

Retrieval-only operations — search, status, sources, analytics, boolean_ask — consumed no generation tokens and draw no generation credit. Only ask, chat, code and the agent paths are billed for generation.

chat is the outlier: roughly three times the latency of ask and just over twice the tokens, because it carries conversation context. Worth knowing before putting it on a user-facing path.

Concurrency. Two, four and eight parallel ask requests all completed successfully. The eight-way run finished in 5.6 s against a 4.3 s serial mean. Rate limiting begins at 120 requests per minute per token.

Token cost against full-context prompting

The alternative most teams start from is pasting everything into the prompt, which pays for the whole corpus on every request. Retrieval-first pays only for the passages an answer actually needs.

The scenario below is 50,000 answers per month at 800 output tokens each, comparing a 100,000-token full-context prompt with a 5,000-token retrieval-first prompt. Prices are each provider's published API token pricing. This is a model, not a measurement.

Model	Full context	Retrieval-first	Reduction
OpenAI GPT-5.5	$26,200	$2,450	90.6%
OpenAI GPT-5.4	$13,100	$1,225	90.6%
OpenAI GPT-5.4 mini	$3,930	$368	90.6%
Claude Opus 4.7	$26,000	$2,250	91.3%
Claude Sonnet 4.6	$15,600	$1,350	91.3%
Claude Haiku 4.5	$5,200	$450	91.3%
xAI Grok 4.3	$6,350	$413	93.5%

The reduction tracks each model's input-to-output price ratio, which is why it lands in a range rather than at a single figure: output tokens are unchanged, so models that charge relatively more for output save slightly less.

Retrieval under load

Retrieval cost should not grow with the size of the corpus. In a deterministic replay — seed 1337, 50,000 writes, 10,000 recall and 10,000 ask requests, concurrency 12 — the value-aware lifecycle policy scanned an average of 563 candidates per retrieval against 3,822 for a time-to-live baseline, an 85.3% reduction, and cut ask p95 from 5,544 ms to 1,290 ms.

That is a synthetic workload on a fixed dataset. It shows the retrieval path staying bounded as stored memory grows; it does not predict latency for any particular production workload.

Method and caveats

Latency figures are six samples per operation from a single client in a single region, against one memory with online search enabled. Treat them as an order of magnitude, not a service guarantee.
Generation latency is dominated by the upstream model. The ask and chat rows ran on gpt-5.2 via OpenAI.
The token-cost table is a model built from published provider pricing and the stated prompt sizes. Your prompt sizes will differ.
The retrieval replay is measured but synthetic: a controlled comparison between lifecycle policies, not a production trace.
Retrieval improves grounding; it does not guarantee correctness. Answers still need evaluating against your own data.

Measure it yourself. Every figure in the latency table comes from calling the documented endpoints with a project service token. The API reference has the exact requests, and each response carries its own usage token counts, so you can measure your own workload rather than trusting ours.