Cache Billing

GPT, Claude, and Gemini series support Prompt Caching: when repeated long prefixes (such as system prompts and long documents) hit the cache, that portion of the input is billed at a cached multiplier discount, and latency also drops significantly.

How to Hit the Cache

• Put fixed content at the very beginning of the message (system prompts, reference documents), and place changing user input at the end;
• Repeating the same prefix within a short period will hit the cache, with no additional parameters required;
• There is a minimum length threshold for prefixes (usually around 1K tokens); if too short, no cache will be created.

How to Confirm a Hit

The cache field in the response usage:

{
  "usage": {
    "prompt_tokens": 5210,
    "completion_tokens": 84,
    "prompt_tokens_details": { "cached_tokens": 4864 }
  }
}

cached_tokens is the input portion that hit the cache; this portion is billed at the cached multiplier (far below the normal input price), and the remaining input is billed at the standard price. See the pricing tags in the console Model Marketplace for the cache multipliers of each model.

Cost-Saving Practices

• For customer service/Agent applications, put all tool definitions and rule documents first; the cost of long sessions can be significantly reduced;
• For batch tasks querying the same set of documents, group by document and send requests continuously to improve hit rates;
• In the console “Logs”, verify the actual charge for each call to confirm that caching is working.