CachingVerifier

A Verifier that remembers verdicts, so the same pair of prompts is judged once instead of on every lookup.

A verifier is the expensive step — usually a model call. Without memoization a hot near miss pays for it every time: if "reset my password" keeps landing next to a cached "reset my PIN", the delegate is asked "same answer?" on every single lookup, always to hear the same "no". This wraps a delegate and caches its answer per (query, cachedPrompt) pair, bounded and optionally expiring.

val verifier = Verifier { q, cached, _ -> model.judgesSameAnswer(q, cached) }.caching(ttl = 6.hours)
val cache = SemanticCache(embedder, verifier = verifier)

The verdict is assumed to be a function of the two prompts alone — not of similarity, which is fixed once the two vectors are — so a staged verifier that keys its decision on the similarity band should not be wrapped. A delegate that throws is never cached: the exception propagates, and SemanticCache treats it as a fail-closed rejection, so a transient outage cannot freeze a "reject" verdict into the cache. Only a real true/false answer is stored.

Safe to share across coroutines: every read and write takes a mutex, and the delegate is called outside the lock so one slow verification never blocks the others.

Parameters