getOrPutStreaming

A getOrPut for streaming completions: caches the assembled text of a streamed answer and replays it on a hit, so a streaming caller never has to fall back to the blocking path.

On a hit the returned Flow emits the cached answer (as a single element — kmemo caches the text, not the original chunk boundaries). On a miss the flow from compute is passed straight through to the collector chunk by chunk while being accumulated, and the assembled text is written to the cache only if the stream completes normally — a stream that fails or is cancelled midway caches nothing, so a partial answer is never served later.

The returned flow is cold: nothing is computed or streamed until it is collected, and the embedding + lookup are done once, up front. Concurrent-miss coalescing and write-behind do not apply to this path. If the embedder throws under EmbedFailurePolicy.FALL_BACK_TO_COMPUTE, the raw stream from compute is returned uncached.

cache.getOrPutStreaming(prompt) { llm.completeStreaming(it) }.collect { chunk -> print(chunk) }