SemanticCache

turns prompts into vectors; you supply it.

where entries live and how they expire. Defaults to a bounded in-memory store.

minimum cosine similarity for a candidate to be considered at all. The default is deliberately tight; calibrate it against your own model with dev.kmemo.calibration.ThresholdCalibrator rather than guessing.

vetoes applied to candidates that clear threshold. MatchGuards.standard by default; MatchGuards.none reproduces the naive similarity-only behaviour.

optional final check, typically a cheap model call, run only on candidates that already passed everything else.

optional cap on a single Verifier.verify call. On timeout — or if the verifier throws — the candidate is rejected, not served: a check that could not complete must fail closed, since the verifier exists precisely to keep an unconfirmed answer out. null (the default) applies no timeout.

how many nearest entries to consider. Looking past the closest one matters: when a guard rejects the top candidate, the second may still be a correct answer.

whether concurrent getOrPut calls for the same prompt in the same scope wait for the first one instead of each calling the model. On by default: a cold cache under load is the case where duplicate calls are most expensive and most likely.

what getOrPut does when the Embedder throws — propagate (the default) or fall back to compute so a lookup is never worse than no cache. See EmbedFailurePolicy. lookup, get and put have no fallback and always propagate. CancellationException always propagates.

when positive, turns on a bounded negative cache: the embedding of a prompt that just missed is remembered, so an immediate repeat of the same brand-new prompt is embedded once rather than once per caller. Extends the concurrent-miss coalescing to the near-in-time sequential case. It only ever reuses an embedding — it never suppresses the store search — so it cannot cause a false hit. 0 (the default) keeps it off.

how long a remembered miss stays usable when negativeCacheSize is positive, or null to keep it until evicted. A short TTL is the point: it should cover a burst, not pin a stale embedding for a prompt that has since been answered elsewhere.

observers notified of every hit, miss and write as it happens (see CacheEvent). Empty by default, and an empty list is free: with no listeners the cache builds no events and measures no latencies, so the hot path is exactly as it was. Each listener runs inline and must be fast and non-throwing — see CacheListener.

when non-null, turns on write-behind: on a getOrPut miss the cache returns as soon as compute does and the store write is applied off the caller's critical path, by a single worker running on this scope. Writes are applied in submission order while buffered; if the buffer is full the write falls through synchronously rather than being dropped, so a write is never lost (only, rarely, reordered under saturation). The window between compute and write is a small chance of a duplicate compute for the same brand-new prompt. Cancel this scope to stop the worker. null (the default) writes through synchronously, and put/warm always do.

how many pending writes to buffer before falling through to a synchronous write. Only meaningful when writeBehindScope is set.

time source for entry timestamps.