Savings

data class Savings(val prices: TokenPrices, val hits: Long, val inputTokens: Long, val outputTokens: Long, val hitsMissingTokenCounts: Long = 0)

What one scope's hits did not cost, with everything the figure rests on.

The inputs travel with the number on purpose. A saving with no price per token attached is a marketing claim rather than a measurement, which is the failure this whole library is against, so there is no way to read amount without also having prices in hand.

Parameters

prices

exactly what the caller declared for this scope.

hits

hits served in this scope since the cache was created. Every one of them is a model call that was not made; a hit is the only event that adds to a saving, so a cache that is filling up reports nothing until it starts answering.

inputTokens

prompt tokens read from the served entries' metadata, summed.

outputTokens

completion tokens read from the served entries' metadata, summed.

hitsMissingTokenCounts

hits whose entry carried neither token count. A number close to hits means the metadata keys are not the ones being written, and amount is then only the flat per-call charge rather than the saving. That is the one way this figure can be quietly wrong, so it is reported next to it rather than left to be discovered.

Constructors

Link copied to clipboard
constructor(prices: TokenPrices, hits: Long, inputTokens: Long, outputTokens: Long, hitsMissingTokenCounts: Long = 0)

Properties

Link copied to clipboard

What those hits would have cost, in currency.

Link copied to clipboard

The unit amount is in, as the caller declared it.

Link copied to clipboard
val hits: Long
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard