Documentation

Response caching

Repeated, identical requests don't need to hit a model twice. Add acache control to a /v1/chat/completions request and BharatRouter will store the response and serve an identical follow-up straight from cache — no upstream call, near-zero latency, and billed at ₹0. It's our parity for Portkey's "simple cache".

Opt-in only — privacy first. Caching stores your response content, so it is a deliberate departure from BharatRouter's zero-retention default. Nothing is ever cached unlessyou ask for it on that specific request. See Privacy & retention.

Turn it on for a request

Send a cache object alongside your normal fields. ttl_seconds is how long the entry lives (default 300s, max 24h):

curl https://api.bharatrouter.com/v1/chat/completions \
  -H "Authorization: Bearer br-..." -H "Content-Type: application/json" \
  -d '{
    "model": "glm-4.6",
    "messages": [{"role": "user", "content": "Summarise the DPDP Act in one line."}],
    "cache": { "mode": "simple", "ttl_seconds": 600 }
  }'

Shorthand: "cache": true is the same as mode simple at the default TTL. The cache field is a BharatRouter control — it is stripped from the payload we forward upstream.

Field	Default	Meaning
`mode`	`simple`	`simple` = exact-match cache (same org, model & request → same answer). `semantic` = embedding-similarity cache (see below).
`ttl_seconds`	`300`	Entry lifetime in seconds (1–86400). Keep it short — a cached answer is stale the moment the world moves on.
`threshold`	`0.92`	Semantic only. Minimum cosine similarity (0–1] for a near-neighbour to count as a hit. Higher = stricter (closer paraphrase required).

Semantic caching

A simple cache only hits on a byte-identical request. Semanticcaching instead embeds your prompt and replays a stored answer when a recent, similarprompt from your org is close enough — so paraphrases ("Summarise the DPDP Act briefly" vs "Give me a one-line summary of the DPDP Act") can share one answer:

{
    "model": "glm-4.6",
    "messages": [{"role": "user", "content": "Give me a one-line summary of the DPDP Act."}],
    "cache": { "mode": "semantic", "threshold": 0.95, "ttl_seconds": 600 }
  }

Your prompt is embedded with a first-party, India-resident model and compared by cosine similarity against your org's recent window for that model. A match at/abovethreshold is replayed; otherwise the request routes normally and its answer joins the window. Everything else — opt-in, org-scoping, short TTL, honest ₹0 accounting, fail-open — is identical to the simple cache. Tune threshold deliberately: too low and you risk a subtly-wrong paraphrase answer; the default is intentionally strict.

Hit or miss — the `x-br-cache` header

Every opted-in response carries an x-br-cache header so you can see what happened:

x-br-cache: miss — served upstream and (if eligible) stored for next time.
x-br-cache: hit — served from the simple exact-match cache; no model was called and the request cost ₹0.
x-br-cache: hit-semantic — served from the semantic cache on a near-neighbour match; the matched cosine similarity is reported in x-br-cache-similarity. Cost ₹0.

A hit replays the original completion verbatim, including its x-br-provider,x-br-model and x-br-residency.

How the cache key is built

An entry is keyed on a hash of your organisation, the model, and a normalised copy of the request body (messages, temperature,max_tokens, stop, tools, and the other output-shaping fields). Two things follow:

Org-scoped. The org id is part of the key, so one tenant's cached content isnever served to another.
Order-insensitive. Fields are canonicalised before hashing, so a client that serialises JSON keys in a different order still gets the same hit.

Routing controls (provider, optimize, fallbacks, …) arenot part of the key — they steer how we route, not what a model returns for a given input.

What is (and isn't) cacheable

Even with a cache control, a request is only cached when it is safe and deterministic-ish to do so:

Non-streaming only. stream: true is never cached.
Successful only. Only 2xx completions with real content are stored — errors are never cached (so a transient upstream failure can't be pinned), and neither are empty/degraded responses.
Not per-request BYOK. Traffic that carries a per-requestupstream_key is never cached — that's your own secret, quota and billing, and its content isn't ours to persist.
Randomised requests. A high temperature makes a poor cache candidate; the explicit per-request cache control is your override — if you ask to cache it, we honour it.

Accounting stays honest

A cache hit is still logged as a real, attributable request in your Activity — it simply showscost ₹0, is marked cached, and accrues no budget spend(no upstream call happened). Token counts from the original completion are preserved so your usage breakdown still adds up.

Privacy & retention

BharatRouter is zero-retention by default: we don't store prompt or completion content. The response cache is the one place that deliberately does — so it is strictlyopt-in per request, never a default, and always visible via thex-br-cache header.

Where. Entries live in BharatRouter's in-region Redis, keyed by an opaque hash (or, for semantic mode, a per-org vector window that also stores the prompt embedding) — never written to disk logs.
How long. Only until ttl_seconds elapses (max 24h), then Redis evicts them automatically.
Who. Org-scoped — only your organisation can ever read your cached answers.
Off switch. Simply omit the cache control and nothing is stored. Operators can also disable the feature fleet-wide.

Because caching stores content, don't opt-in for requests whose responses must not be retained even briefly (e.g. content under a strict data-handling agreement) — leave those on the default zero-retention path.

Fail-open by design

The cache is an optimisation, never a dependency. If Redis is unavailable or a cache read/write errors, the request falls straight through to normal routing — a cache problem can never fail an inference call.