Package-level declarations

Types

class CacheEntry(val id: String, val scope: String, val prompt: String, val response: String, embedding: FloatArray, val createdAt: Instant, val metadata: Map<String, String> = emptyMap())

One cached prompt/response pair plus the vector used to find it again.

CacheEvent

sealed interface CacheEvent

Something a SemanticCache (or its CacheStore) did, delivered to a CacheListener the moment it happens.

CacheEvents

class CacheEvents(bufferCapacity: Int = DEFAULT_BUFFER_CAPACITY) : CacheListener

A CacheListener that republishes events as a Flow you can collect.

CacheExplanation

class CacheExplanation(val prompt: String, val scope: String, val threshold: Double, val candidates: List<CandidateTrace>)

A read-only trace of how SemanticCache.explain evaluated one prompt — the tool you reach for when a hit you expected did not happen, or one you did not expect did.

CacheListener

fun interface CacheListener

A sink for CacheEvents, called inline the moment each one happens.

CacheLookup

sealed interface CacheLookup

Outcome of SemanticCache.lookup.

CacheStats

data class CacheStats(val lookups: Long, val hits: Long, val misses: Long, val belowThreshold: Long, val guardRejections: Long, val verifierRejections: Long, val writes: Long, val guardRejectionsByGuard: Map<String, Long> = emptyMap())

Counters for one SemanticCache instance, from creation to now.

CacheStore

interface CacheStore

Storage and nearest-neighbour search behind a SemanticCache.

CachingVerifier

class CachingVerifier(delegate: Verifier, maxEntries: Int = DEFAULT_MAX_ENTRIES, ttl: Duration? = null, clock: Clock = Clock.systemUTC()) : Verifier

A Verifier that remembers verdicts, so the same pair of prompts is judged once instead of on every lookup.

CandidateTrace

class CandidateTrace(val prompt: String, val similarity: Double, val aboveThreshold: Boolean, val guardVerdicts: Map<String, GuardVerdict>)

One candidate entry and every guard's verdict on it, from a CacheExplanation.

Embedder

fun interface Embedder

Turns text into a vector. kmemo ships no embedding implementation on purpose: you bring your own (OpenAI, Cohere, Voyage, a local ONNX model, whatever your stack already pays for), and kmemo stays free of provider SDKs.

EmbedFailurePolicy

enum EmbedFailurePolicy : Enum<EmbedFailurePolicy>

What SemanticCache.getOrPut does when the Embedder throws.

EventTimings

class EventTimings(val embedNanos: Long, val searchNanos: Long, val verifierNanos: Long)

Wall-clock nanoseconds spent in each stage of a single lookup, for latency metrics.

EvictionCause

enum EvictionCause : Enum<EvictionCause>

Why a CacheEvent.Eviction happened.

MissReason

enum MissReason : Enum<MissReason>

Why a CacheLookup.Miss happened.

ResponseCodec

interface ResponseCodec<T>

Turns a typed response into the String a SemanticCache stores, and back.

RetryingEmbedder

class RetryingEmbedder(delegate: Embedder, maxAttempts: Int = DEFAULT_MAX_ATTEMPTS, initialDelay: Duration = DEFAULT_INITIAL_DELAY, maxDelay: Duration = DEFAULT_MAX_DELAY, factor: Double = DEFAULT_FACTOR, jitter: Double = DEFAULT_JITTER, retryOn: (Throwable) -> Boolean = { true }) : Embedder

An Embedder that retries a failing delegate with exponential backoff and jitter.

ScoredEntry

class ScoredEntry(val entry: CacheEntry, val similarity: Double)

A CacheEntry together with its similarity to the query vector, in [-1.0, 1.0].

SemanticCache

class SemanticCache(embedder: Embedder, store: CacheStore = InMemoryStore(), threshold: Double = DEFAULT_THRESHOLD, guards: List<MatchGuard> = MatchGuards.standard(), verifier: Verifier? = null, verifierTimeout: Duration? = null, candidates: Int = DEFAULT_CANDIDATES, coalesceConcurrentMisses: Boolean = true, embedFailurePolicy: EmbedFailurePolicy = EmbedFailurePolicy.PROPAGATE, negativeCacheSize: Int = 0, negativeCacheTtl: Duration? = null, listeners: List<CacheListener> = emptyList(), writeBehindScope: CoroutineScope? = null, writeBehindCapacity: Int = DEFAULT_WRITE_BEHIND_CAPACITY, clock: Clock = Clock.systemUTC())

A cache keyed by what a prompt means rather than by its exact bytes.

SemanticCacheBuilder

class SemanticCacheBuilder(embedder: Embedder)

Builds a SemanticCache with a DSL instead of a long positional constructor call.

Vectors

object Vectors

Vector maths shared by SemanticCache and by CacheStore implementations.

Verifier

fun interface Verifier

Last line of defence before a cached response is served.

WarmEntry

class WarmEntry(val prompt: String, val response: String, val scope: String = SemanticCache.DEFAULT_SCOPE, val metadata: Map<String, String> = emptyMap())

One prompt/response pair to preload into a SemanticCache via SemanticCache.warm.

Functions

caching

fun Verifier.caching(maxEntries: Int = CachingVerifier.DEFAULT_MAX_ENTRIES, ttl: Duration? = null, clock: Clock = Clock.systemUTC()): CachingVerifier

Wraps this verifier in a CachingVerifier that memoizes verdicts per (query, cachedPrompt) pair.

catching

inline fun <T> catching(block: () -> T): Result<T>

Runs block and captures its outcome as a Result, the coroutine-safe way.

retrying

fun Embedder.retrying(maxAttempts: Int = RetryingEmbedder.DEFAULT_MAX_ATTEMPTS, initialDelay: Duration = RetryingEmbedder.DEFAULT_INITIAL_DELAY, maxDelay: Duration = RetryingEmbedder.DEFAULT_MAX_DELAY, factor: Double = RetryingEmbedder.DEFAULT_FACTOR, jitter: Double = RetryingEmbedder.DEFAULT_JITTER, retryOn: (Throwable) -> Boolean = { true }): RetryingEmbedder

Wraps this embedder in a RetryingEmbedder that retries transient failures with jittered backoff.

semanticCache

fun semanticCache(embedder: Embedder, configure: SemanticCacheBuilder.() -> Unit = {}): SemanticCache

Builds a SemanticCache with the SemanticCacheBuilder DSL.