Token caching starts with the prefix

Understand what gets reused

DeepSeek’s documentation describes automatic input caching. Persisted prefixes are complete units; a subsequent request must fully match one of those units to obtain the corresponding hit. The same paragraph appearing elsewhere in a prompt is not enough.

This cache does not store a finished answer to return unchanged. The model still generates the output. It therefore serves a different purpose from an application cache, which can return a previously computed result when your own service’s cache key matches. Decide which kind of reuse the workload actually needs before optimizing either layer.

Put stable content first

Place stable elements at the beginning: general instructions, conventions, the output schema and a shared document. Leave the changing question and request-specific details toward the end. Avoid timestamps or random identifiers before shared material unless the task actually needs them.

This structure also helps maintenance. A versioned template makes it easier to explain why a change affected model behavior. Do not sacrifice correctness in pursuit of cache hits, however. If a document has changed, the prompt must contain the new version, even when that temporarily reduces reuse. A cheaper answer based on outdated context is not an optimization.

One document, several questions

Consider an application answering questions about one manual. Early requests share instructions and the manual text, while the question changes. The service can detect and persist the common portion; later requests may reuse it under the documented rules. Do not assume the second question will always receive a hit.

Stable prompt:
  instructions + manual version + manual text

Variable portion:
  user question + the format needed for this task

Measurement:
  hit_ratio = hit_tokens / (hit_tokens + miss_tokens)

For long conversations, retain the history that is genuinely needed and preserve its order. Changing an early message changes the prefix. Summarizing the history may reduce total tokens while losing some cache hits. Compare final spend, not an isolated cache percentage.

Read the actual usage fields

Save prompt_cache_hit_tokens and prompt_cache_miss_tokens from usage. Aggregate them by template version and task family. Do not estimate caching solely by comparing two strings; the provider’s observed usage is the useful record for reconstructing consumption.

Alongside hit ratio, report total input, output and cost per useful request. An unnecessarily long prompt can achieve an excellent hit ratio while costing more than a short one. Monitoring should flag both a fall in hits and an increase in total volume: they have different causes and require different fixes. Look at the numbers together before changing the template.

Calculate the full cost

Input cost is (hit_tokens × hit_rate + miss_tokens × miss_rate) / 1,000,000. Add output separately. Use the actual model and pricing window. Applying one endpoint’s discounted rate to another endpoint creates an artificial comparison.

When part of the input gets cheaper, the reduction in the total bill depends on how much output contributes. An application generating long articles may see a modest overall benefit despite many cache hits. An extraction workflow with a long document and a short answer can look quite different. Report the workload assumptions whenever showing a savings estimate.

Treat it as best effort

The cache is best effort. It should not become a correctness requirement or a guarantee of minimum spend. Prepare one budget scenario with no hits and another based on observed hits. The difference is an optimization opportunity, not a resource on which the product’s availability should depend.

The verification protocol is straightforward: fix the template, send representative requests, inspect usage and repeat after a change. Deepweight separates executed measurements from explanations of a mechanism. This article provides the method; it does not claim a proprietary savings percentage that has never been recorded in a documented run.