RetryingEmbedder
An Embedder that retries a failing delegate with exponential backoff and jitter.
The embedder is a network call on every lookup, so a transient blip — a dropped connection, a 429, a provider hiccup — should not become a cache miss (or, under EmbedFailurePolicy.PROPAGATE, a caller-visible error) when a second attempt a moment later would have succeeded. This wraps any Embedder and re-issues embed / embedAll up to maxAttempts times, backing off between tries.
val embedder = Embedder { text -> openAi.embed(text) }.retrying(maxAttempts = 4)
val cache = SemanticCache(embedder)Opt-in on purpose: the default cache issues exactly one embed call per lookup, and retries — which add latency and can amplify load on a provider that is already struggling — are a decision the integrator makes, not a default paid by everyone. Retrying is transparent to SemanticCache; it is just an Embedder, so it composes with everything else (including a batch endpoint override).
A CancellationException is never retried and never delayed — it propagates immediately, so coroutine cancellation is unaffected. Which other throwables are worth retrying is the caller's call via retryOn; the default retries anything that is not a cancellation, on the assumption that an embedder either has an idempotent read or has been made safe to call twice.
Parameters
the embedder whose calls are retried.
total attempts including the first, so 1 disables retrying. Must be positive.
backoff before the second attempt; each further wait multiplies by factor.
ceiling on a single backoff wait, so exponential growth cannot run away.
multiplier applied to the delay after each failed attempt (2.0 doubles it).
fraction of each computed delay that is randomized away, in [0.0, 1.0]. 0.0 waits the exact backoff; the default 0.5 waits a random span in [0.5, 1.0] of it, so a fleet whose calls all failed together does not retry in lockstep and hammer the provider on the same tick.
decides whether a given failure is worth another attempt. Returning false re-throws immediately. CancellationException is excluded before this is ever consulted.
Functions
Embeds a single text. Vectors need not be normalized; kmemo normalizes on the way in.
Wraps this embedder in a RetryingEmbedder that retries transient failures with jittered backoff.