I make a first LLM call with more than 1024 input tokens with `prompt_cache_key=abc`.

I made a second LLM call with more than 1024 input tokens with ‘prompt_cache_key=abc’.

Both LLM calls have strictly the same system_instructions. From my understanding, the second call will reuse the cache made by the first call, and it will show `cached_tokens` > 0 (= 0 for the first one).

Now, suppose the second LLM call has ‘prompt_cache_key=abcZ’. Can I expect it to not land on the same machine as the first call? Both calls outputs should yield `cached_tokens` = 0.

Now suppose I make `n` calls, all with the same prefix longer than 1024, each call having its own unique and distinct `prompt_cache_key`. Can I expect all of these calls to not share any cache? In other words, `cached_tokens` = 0 in the token usage stats for all of them?

Here is our presentation of information about routing, so let’s dissect the exact wording:

• Requests are routed to a machine based on a hash of the initial prefix of the prompt. The hash typically uses the first 256 tokens, though the exact length varies depending on the model.
• If you provide the prompt_cache_key parameter, it is combined with the prefix hash, allowing you to influence routing and improve cache hit rates. This is especially beneficial when many requests share long, common prefixes.

So that gives us:

• there is a “prefix hash”, so of we say “prefix”, it means the initial hash one based on prompt, not the potentially combined output of prefix and hash. Then we look after routing is performed

Cache Lookup: The system checks if the initial portion (prefix) of your prompt exists in the cache on the selected machine.

So what we have is a routing layer.

Consider if OpenAI only had one inference server instead of a great many - with the above mechanic you would always receive a cache hit if one persists, regardless of prompt_cache_key that is sent.

Then we continue with some more information:

• On GPT-5.6 models and later model families, you must set prompt_cache_key to use the more reliable matching for both implicit and explicit caching. At each breakpoint, the service matches the key with the exact prompt prefix. Without a key, requests may still receive automatic cache hits, but they do not use the improved matching.

That says “must”. However, it does not define “improved matching”. It also doesn’t exclude a match with a differering prefix.

Conclusion: So it seem to me there is no absolute promise that you are denied matches with differing prompt cache key - only that they become much more unlikely, and would serve as a reroute mechanism to lessen the chance of a match when there is any alternate destination server without hash collision in the hash subspace.

What is true about this description of breakpoints that must be further discovered:

• exact breakpoint: if you don’t “mark” past breakpoints again in a new explicit API request, they cannot be matched.

@_j Thanks for your reply. I forgot to specify that I am using GPT-5.2, but I guess your observations would still apply?

I wanted to know whether the `prompt_cache_key` would suffice to isolate caches. I tried 20 calls (same prefix everywhere with `gpt-5.2-2025-12-11`, `reasoning={“effort”: “low”}`, and 1.5 seconds of sleep between calls) successively with distinct unique keys. 9 out of 20 calls (~45%) hit the cache despite having a distinct key.

I did the same experiment again a second time more than 1 hour later. I had set `prompt_cache_retention=in_memory`, so that the cache from the first run survives at most 1 hour and cannot be used for the second run. The hit cache rate amount to 40%. The `prompt_cache_key`s are all different with respect to the previous run.

The only reliable way I found so far to force cache isolation was to inject a random hash within the first 128 tokens of the prompt (inside a tool’s description), which changes the prefix itself rather than relying on prompt_cache_key.