CacheStats

data class CacheStats(val lookups: Long, val hits: Long, val misses: Long, val belowThreshold: Long, val guardRejections: Long, val verifierRejections: Long, val writes: Long, val guardRejectionsByGuard: Map<String, Long> = emptyMap())

Counters for one SemanticCache instance, from creation to now.

The breakdown of misses is the part worth watching. A high belowThreshold means your traffic repeats less than you assumed, or your threshold is too tight for your embedding model. A high guardRejections means the opposite: prompts are landing close together and the guards are catching near-misses your threshold alone would have served.

Constructors

Link copied to clipboard
constructor(lookups: Long, hits: Long, misses: Long, belowThreshold: Long, guardRejections: Long, verifierRejections: Long, writes: Long, guardRejectionsByGuard: Map<String, Long> = emptyMap())

Properties

Link copied to clipboard

Misses where the best candidate scored below the threshold.

Link copied to clipboard

Fraction of lookups that reached a guard and were stopped there, in [0.0, 1.0].

Link copied to clipboard

Misses where a guard vetoed a candidate that had cleared the threshold.

Link copied to clipboard
Link copied to clipboard

Fraction of lookups served from cache, in [0.0, 1.0]. 0.0 when nothing has been looked up.

Link copied to clipboard
val hits: Long

Lookups that served a cached response.

Link copied to clipboard
Link copied to clipboard

Lookups that did not. Always lookups - hits.

Link copied to clipboard

Misses where the verifier vetoed a candidate that had cleared threshold and guards.

Link copied to clipboard

Entries written through SemanticCache.put or SemanticCache.getOrPut.