Package-level declarations

Types

class AdaptiveThresholds(floor: Double, ceiling: Double, targetRejectionRate: Double = DEFAULT_TARGET_REJECTION_RATE, minimumSamples: Int = DEFAULT_MINIMUM_SAMPLES, step: Double = DEFAULT_STEP) : CacheListener

Moves each scope's similarity threshold towards the value its own traffic justifies.

AdmissionPolicy

class AdmissionPolicy(val minSightings: Int = DEFAULT_MIN_SIGHTINGS, val sketchWidth: Int = DEFAULT_SKETCH_WIDTH, val sketchDepth: Int = DEFAULT_SKETCH_DEPTH, val resetAfter: Int = DEFAULT_RESET_AFTER)

Decides whether a prompt has been asked often enough to be worth storing.

CacheEntry

class CacheEntry(val id: String, val scope: String, val prompt: String, val response: String, embedding: FloatArray, val createdAt: Instant, val metadata: Map<String, String> = emptyMap(), val tags: Set<String> = emptySet(), val embedder: String = Embedder.UNDECLARED, val chunkLengths: List<Int> = emptyList())

One cached prompt/response pair plus the vector used to find it again.

CacheEvent

sealed interface CacheEvent

Something a SemanticCache (or its CacheStore) did, delivered to a CacheListener the moment it happens.

CacheEvents

class CacheEvents(bufferCapacity: Int = DEFAULT_BUFFER_CAPACITY) : CacheListener

A CacheListener that republishes events as a Flow you can collect.

CacheExplanation

class CacheExplanation(val prompt: String, val scope: String, val threshold: Double, val candidates: List<CandidateTrace>)

A read-only trace of how SemanticCache.explain evaluated one prompt — the tool you reach for when a hit you expected did not happen, or one you did not expect did.

CacheListener

fun interface CacheListener

A sink for CacheEvents, called inline the moment each one happens.

CacheLookup

sealed interface CacheLookup

Outcome of SemanticCache.lookup.

CachePolicy

fun interface CachePolicy

Decides whether a prompt and its computed response may be persisted at all.

CacheStats

data class CacheStats(val lookups: Long, val hits: Long, val misses: Long, val belowThreshold: Long, val guardRejections: Long, val verifierRejections: Long, val writes: Long, val guardRejectionsByGuard: Map<String, Long> = emptyMap(), val degradedLookups: Long = 0, val writesVetoed: Long = 0, val exactHits: Long = 0, val embedderMismatches: Long = 0, val savings: Map<String, Savings> = emptyMap(), val writesNotAdmitted: Long = 0)

Counters for one SemanticCache instance, from creation to now.

CacheStore

interface CacheStore

Storage and nearest-neighbour search behind a SemanticCache.

CachingVerifier

class CachingVerifier(delegate: Verifier, maxEntries: Int = DEFAULT_MAX_ENTRIES, ttl: Duration? = null, clock: Clock = Clock.System) : Verifier

A Verifier that remembers verdicts, so the same pair of prompts is judged once instead of on every lookup.

CandidateReranker

fun interface CandidateReranker

Reorders the candidates that cleared the similarity threshold, before the guards see them.

CandidateTrace

class CandidateTrace(val prompt: String, val similarity: Double, val aboveThreshold: Boolean, val guardVerdicts: Map<String, GuardVerdict>, val embedderMatches: Boolean = true)

One candidate entry and every guard's verdict on it, from a CacheExplanation.

ConfidenceVerifier

interface ConfidenceVerifier : Verifier

A Verifier that reports how sure it is, so the caller sets where the line falls.

DegradedOperation

enum DegradedOperation : Enum<DegradedOperation>

Which entry point a CacheEvent.Degraded came from.

Embedder

fun interface Embedder

Turns text into a vector. kmemo ships no embedding implementation on purpose: you bring your own (OpenAI, Cohere, Voyage, a local ONNX model, whatever your stack already pays for), and kmemo stays free of provider SDKs.

EmbedFailurePolicy

enum EmbedFailurePolicy : Enum<EmbedFailurePolicy>

What SemanticCache.getOrPut does when the Embedder throws.

EntryCipher

interface EntryCipher

Encrypts what a CacheStore persists, so a database full of user questions is not what an auditor finds.

EventTimings

class EventTimings(val embedNanos: Long, val searchNanos: Long, val verifierNanos: Long)

Wall-clock nanoseconds spent in each stage of a single lookup, for latency metrics.

EvictionCause

enum EvictionCause : Enum<EvictionCause>

Why a CacheEvent.Eviction happened.

MissReason

enum MissReason : Enum<MissReason>

Why a CacheLookup.Miss happened.

MmrReranker

class MmrReranker(lambda: Double = DEFAULT_LAMBDA) : CandidateReranker

Maximal Marginal Relevance: prefers a candidate that is close to the query and unlike the candidates already chosen.

PolicyVerdict

sealed interface PolicyVerdict

The outcome of a CachePolicy decision.

Quantization

enum Quantization : Enum<Quantization>

How a CacheStore may compress vectors for the scan only.

ResponseCodec

interface ResponseCodec<T>

Turns a typed response into the String a SemanticCache stores, and back.

RetryingEmbedder

class RetryingEmbedder(delegate: Embedder, maxAttempts: Int = DEFAULT_MAX_ATTEMPTS, initialDelay: Duration = DEFAULT_INITIAL_DELAY, maxDelay: Duration = DEFAULT_MAX_DELAY, factor: Double = DEFAULT_FACTOR, jitter: Double = DEFAULT_JITTER, retryOn: (Throwable) -> Boolean = { true }) : Embedder

An Embedder that retries a failing delegate with exponential backoff and jitter.

Savings

data class Savings(val prices: TokenPrices, val hits: Long, val inputTokens: Long, val outputTokens: Long, val hitsMissingTokenCounts: Long = 0)

What one scope's hits did not cost, with everything the figure rests on.

ScoredEntry

class ScoredEntry(val entry: CacheEntry, val similarity: Double)

A CacheEntry together with its similarity to the query vector, in [-1.0, 1.0].

SemanticCache

class SemanticCache(embedder: Embedder, store: CacheStore = InMemoryStore(), threshold: Double = DEFAULT_THRESHOLD, guards: List<MatchGuard> = MatchGuards.standard(), verifier: Verifier? = null, verifierTimeout: Duration? = null, candidates: Int = DEFAULT_CANDIDATES, coalesceConcurrentMisses: Boolean = true, embedFailurePolicy: EmbedFailurePolicy = EmbedFailurePolicy.PROPAGATE, negativeCacheSize: Int = 0, negativeCacheTtl: Duration? = null, listeners: List<CacheListener> = emptyList(), writeBehindScope: CoroutineScope? = null, writeBehindCapacity: Int = DEFAULT_WRITE_BEHIND_CAPACITY, clock: Clock = Clock.System, cachePolicy: CachePolicy? = null, exactCacheSize: Int = 0, exactCacheTtl: Duration? = null, thresholds: Map<String, Double> = emptyMap(), shadowThresholds: List<Double> = emptyList(), reranker: CandidateReranker? = null, deduplicateWrites: Double? = null, adaptiveThresholds: AdaptiveThresholds? = null, prices: Map<String, TokenPrices> = emptyMap(), admissionPolicy: AdmissionPolicy? = null, requireTenant: Boolean = false)

A cache keyed by what a prompt means rather than by its exact bytes.

SemanticCacheBuilder

class SemanticCacheBuilder(embedder: Embedder)

Builds a SemanticCache with a DSL instead of a long positional constructor call.

ShadowDecision

class ShadowDecision(val threshold: Double, val wouldHit: Boolean, val reason: MissReason?, val bestSimilarity: Double?, val matchedPrompt: String?, val guardName: String?)

What one threshold would have decided, and why.

ShadowReport

class ShadowReport(val scope: String, val prompt: String, val decisions: List<ShadowDecision>)

What the cache would have done for one prompt, at each threshold it was asked about.

StreamReplay

enum StreamReplay : Enum<StreamReplay>

How SemanticCache.getOrPutStreaming replays a cached answer on a hit.

TenantedCache

class TenantedCache

A SemanticCache bound to one tenant, through which nothing can reach another tenant's entries.

TokenPrices

data class TokenPrices(val currency: String, val perInputToken: Double = 0.0, val perOutputToken: Double = 0.0, val perCall: Double = 0.0, val inputTokensKey: String = DEFAULT_INPUT_TOKENS_KEY, val outputTokensKey: String = DEFAULT_OUTPUT_TOKENS_KEY)

What one model call in a scope costs, declared by the caller.

Vectors

object Vectors

Vector maths shared by SemanticCache and by CacheStore implementations.

Verifier

fun interface Verifier

Last line of defence before a cached response is served.

WarmEntry

class WarmEntry(val prompt: String, val response: String, val scope: String = SemanticCache.DEFAULT_SCOPE, val metadata: Map<String, String> = emptyMap())

One prompt/response pair to preload into a SemanticCache via SemanticCache.warm.

Functions

caching

fun Verifier.caching(maxEntries: Int = CachingVerifier.DEFAULT_MAX_ENTRIES, ttl: Duration? = null, clock: Clock = Clock.System): CachingVerifier

Wraps this verifier in a CachingVerifier that memoizes verdicts per (query, cachedPrompt) pair.

catching

inline fun <T> catching(block: () -> T): Result<T>

Runs block and captures its outcome as a Result, the coroutine-safe way.

retrying

fun Embedder.retrying(maxAttempts: Int = RetryingEmbedder.DEFAULT_MAX_ATTEMPTS, initialDelay: Duration = RetryingEmbedder.DEFAULT_INITIAL_DELAY, maxDelay: Duration = RetryingEmbedder.DEFAULT_MAX_DELAY, factor: Double = RetryingEmbedder.DEFAULT_FACTOR, jitter: Double = RetryingEmbedder.DEFAULT_JITTER, retryOn: (Throwable) -> Boolean = { true }): RetryingEmbedder

Wraps this embedder in a RetryingEmbedder that retries transient failures with jittered backoff.

semanticCache

fun semanticCache(embedder: Embedder, configure: SemanticCacheBuilder.() -> Unit = {}): SemanticCache

Builds a SemanticCache with the SemanticCacheBuilder DSL.