Quantization

How a CacheStore may compress vectors for the scan only.

The discipline, which is the whole point

A quantized vector is a cheaper approximation, and an approximation in a cache that exists to avoid wrong answers has to be contained. So the rule is absolute: quantization decides which candidates are looked at, never whether one is served. Survivors of the compressed scan are rescored against the full-precision vectors, and the number that meets the threshold, the guards and the verifier is always the exact one.

That confines the error to a recall effect. The worst a bad quantization can do is fail to surface an entry that would have been served — a miss, which costs one API call. It cannot move a similarity across the threshold, because the similarity it produced was thrown away before the threshold was consulted.

The dev.kmemo.store.InMemoryStore applies it; a store backed by a database that quantizes for you should follow the same rule or say that it does not.

Choosing one

NONE until the scan is actually the problem. A linear scan over ten thousand 1,536-dimensional vectors is well under a millisecond, and the network call it replaces is a hundred times that.

Entries

Link copied to clipboard

Full precision. The scan reads the same vectors the decision uses.

Link copied to clipboard

One signed byte per dimension: a quarter of the memory traffic, integer arithmetic in the scan.

Link copied to clipboard

One bit per dimension, the sign: 1/32 of the memory traffic, and the scan becomes popcounts.

Properties

Link copied to clipboard

Returns a representation of an immutable list of all enum entries, in the order they're declared.

Link copied to clipboard
expect val name: String
Link copied to clipboard
expect val ordinal: Int

Functions

Link copied to clipboard

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Link copied to clipboard

Returns an array containing the constants of this enum type, in the order they're declared.