Docs / Runtime
Prefix caching
Repeated input prefixes that hit the cache are billed at 20%.
Hit conditions
- The messages prefix is byte-identical (including the system prompt and ordering).
- The prefix is at least 1024 tokens long.
- The two requests fall within the cache TTL, typically 5 to 60 minutes.
How to raise your hit rate
- Put fixed system prompts and document context first, and the changing user input last.
- Avoid timestamps, random ids, or anything else that changes on every request inside the prompt.
- For RAG, concatenate retrieved passages in a stable order.
Cache-hit requests carry a "cached" marker in the usage logs, and the usage stats page tallies the amount saved separately.
PreviousRate limits
NextThis is the last doc