Prompt caching and cost
Reuse prompt prefixes and verify cache hits through usage fields.
Prompt caching reuses computation for repeated input prefixes, not a previous answer. Verify hits using cached token counts, not matching answers or lower latency.
The OpenAI example uses deepseek-flash without cache-control parameters. On 2026-10-12, its second request reported 25,856 cached tokens from 26,016 input tokens. These are observed values, not a guarantee for every request.
1. Prepare your terminal
Use a Praxis cloud API key in PRAXIS_API_KEY; see API keys and authentication. Do not put the key in source code. Install Python and uv; TypeScript examples use Bun.
test -n "$PRAXIS_API_KEY"Expected: exit code 0. The examples default to https://api.deepraxis.com. To use another environment, set PRAXIS_API_BASE to its API origin, without a trailing slash or API path.
Save each Python block using its indicated filename. Run it with uv run -q --no-project --with openai --with anthropic python <filename>. Each block is independent.
2. Request the same prefix twice
Put stable instructions and reference material first, with the current question last. Keep the model, prefix content, and ordering unchanged. This example generates long demonstration material and makes two billable model calls.
Save as prompt-cache.py. Run uv run -q --no-project --with openai --with anthropic python prompt-cache.py.
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ['PRAXIS_API_KEY'],
base_url=os.environ.get('PRAXIS_API_BASE', 'https://api.deepraxis.com') + '/openai/v1')
prefix = '\n'.join(f'Record {i}: Paris is in France. The demo temperature is 22 Celsius.' for i in range(1500))
hits = []
for turn in range(2):
response = client.chat.completions.create(model='deepseek-flash',
messages=[{'role':'system','content':prefix},
{'role':'user','content':'What country is Paris in? Reply with one word.'}],
reasoning_effort='none', max_tokens=64)
print('X-Request-Id:', response._request_id)
assert response.choices[0].finish_reason == 'stop'
usage = response.usage
assert usage and usage.prompt_tokens_details is not None
cached = usage.prompt_tokens_details.cached_tokens
assert cached is not None
hits.append(cached)
print(turn + 1, 'prompt_tokens:', usage.prompt_tokens, 'cached_tokens:', cached)
assert hits[1] > 0, 'No cache hit observed on the second request'Expected: input and cached token counts for both requests, with cached_tokens > 0 on the second. Exit code: 0. The first request may also hit an existing cache; it need not report zero.
usage.prompt_tokens_details.cached_tokens is a subset of prompt_tokens; do not add it again. Caching reduces eligible input costs, not output costs. Calculate costs using the model's cached-input, ordinary-input, and output rates, rather than a fixed discount. Minimum prefix length and retention depend on the model.
3. Check a hit with TypeScript
Save as prompt-cache.ts. Run bun prompt-cache.ts; no SDK installation is needed.
import assert from 'node:assert/strict';
const base = process.env.PRAXIS_API_BASE ?? 'https://api.deepraxis.com';
const prefix = Array.from({length: 1500}, (_, i) =>
`Record ${i}: Paris is in France. The demo temperature is 22 Celsius.`).join('\n');
const hits: number[] = [];
for (let turn = 0; turn < 2; turn++) {
const response = await fetch(`${base}/openai/v1/chat/completions`, {
method: 'POST', headers: {'Authorization': `Bearer ${process.env.PRAXIS_API_KEY}`, 'Content-Type': 'application/json'},
body: JSON.stringify({model: 'deepseek-flash', max_tokens: 64, reasoning_effort: 'none',
messages: [{role: 'system', content: prefix},
{role: 'user', content: 'What country is Paris in? Reply with one word.'}]}),
});
console.log('X-Request-Id:', response.headers.get('x-request-id'));
const data = await response.json();
assert(response.ok, JSON.stringify(data));
assert.equal(data.choices[0].finish_reason, 'stop');
const cached = data.usage.prompt_tokens_details?.cached_tokens;
assert.equal(typeof cached, 'number');
hits.push(cached);
console.log(turn + 1, 'prompt_tokens:', data.usage.prompt_tokens, 'cached_tokens:', cached);
}
assert(hits[1] > 0, 'No cache hit observed on the second request');Expected: positive cached_tokens on the second request. Exit code: 0.
4. Observe Anthropic cache_control
Messages accepts cache_control: {type: "ephemeral"} on a stable text block. This example uses claude-opus-5 and reads cache_creation_input_tokens and cache_read_input_tokens after each call. Messages input_tokens excludes cache reads and writes; total input is the sum of all three fields.
During testing, both requests were accepted but reported zero cache reads and writes. This example observes cache status; it does not promise a hit. supported_parameters in GET /openai/v1/models exposes parameter declarations, not proof of individual cache hits. Use the returned usage fields as evidence.
Save as prompt-cache-anthropic.py. Run uv run -q --no-project --with openai --with anthropic python prompt-cache-anthropic.py.
import os
from anthropic import Anthropic
client = Anthropic(api_key=os.environ['PRAXIS_API_KEY'],
base_url=os.environ.get('PRAXIS_API_BASE', 'https://api.deepraxis.com') + '/anthropic')
prefix = '\n'.join(f'Record {i}: Paris is in France. The demo temperature is 22 Celsius.' for i in range(1500))
hits = []
for turn in range(2):
raw = client.messages.with_raw_response.create(model='claude-opus-5', max_tokens=64,
system=[{'type':'text','text':prefix,'cache_control':{'type':'ephemeral'}}],
messages=[{'role':'user','content':'What country is Paris in? Reply with one word.'}])
print('X-Request-Id:', raw.headers.get('x-request-id'))
message = raw.parse()
assert message.stop_reason == 'end_turn'
usage = message.usage
assert usage.cache_read_input_tokens is not None
assert usage.cache_creation_input_tokens is not None
hits.append(usage.cache_read_input_tokens)
print(turn + 1, 'input:', usage.input_tokens, 'write:', usage.cache_creation_input_tokens,
'read:', usage.cache_read_input_tokens)
print('HIT' if hits[1] > 0 else 'MISS: no cached input reported')Expected: two completed requests, read/write statistics, and HIT or MISS: no cached input reported. Exit code: 0. Only a positive read count establishes a hit. Exit code 0 proves only successful requests and usage inspection.
Handle the observed miss
The Messages example returned cache_creation_input_tokens=0 and cache_read_input_tokens=0, without an HTTP error. Request acceptance does not establish caching. To reproduce a verified hit, run the deepseek-flash OpenAI example above and check for positive cached_tokens.
To keep prefixes reusable, use the same model and complete prefix for both calls. Avoid timestamps or changing identifiers inside the prefix. Keep usage and X-Request-Id for comparison. Matching answers are not cache evidence.