QuantizedArrays.jl - array quantization and compression.

ArrayQuantizer{Q,U,D,T,N}

The array quantizer object. It transforms an array with elements of type T into a 'quantized' version with elemetns of type U.

Fields

• quantization::Q type of quantization employed i.e. additive, orthogonal
• dims::NTuple{N,Int} the original array dimensionality
• codebooks::Vector{CodeBook{U,T}} the codebooks
• k::Int the number of vector prototypes in each codebooks
• distance::D the distance employed
• rot::Matrix{T} rotation matrix

CodeBook{U<:Unsigned,T}

The codebook structure. It holds the codes corresponding to the vector prototypes and the mapping bethween the codes and prototypes.

Fields

• codes::Vector{U} the codes
• vectors::Matrix{T} the prototypes
• codemap::Dict{U,Int} mapping from code to column in vectors

QuantizedArray{Q<:AbstractQuantization,U<:Unsigned,D<:PreMetric,T,N} <: AbstractArray{T,N}

A quantized array. It represents a 'quantized' representation of an original array, equivalent to a lossy compressed version.

Fields

• quantizer::ArrayQuantizer{Q,U,D,T,N} the quantized of the array
• data::Matrix{U} the actual compressed representation of the quantized array

build_codebooks(X, k, m, U [;method=DEFAULT_METHOD, distance=DEFAULT_DISTANCE, kwargs])

Generates m codebooks of k prototypes each for the input matrix X using the algorithm specified my method and distance distance. Specific codebook aglorithm keyword arguments can be specified as well.

Arguments

• X::AbstractMatrix{T} input matrix of type T
• k::Int number of prototypes/codebook
• m::Int number of codebooks
• U::Type{<:Unsigned} type for codebook codes

Keyword arguments

• method::Symbol the algorithm to be employed for codebook

generation; possible values are :sample (default), :pq for classical k-means clustering codebooks and :opq for 'cartesian' k-means clustering codebooks

• distance::PreMetric the distance to be used in the

codebook generation methods and data encoding

• kwargs... other codebook generation algorithm specific

keyword arguments such as maxiter::Int.

build_quantizer(aa [;kwargs])

Builds an array quantizer using the input array aa.

Keyword arguments

• k::Int the number of vector prototypes in each codebook
• m::Int the number of codebooks
• method::Symbol the algorithm to be employed for codebook

generation; possible values are :sample (default), :pq for classical k-means clustering codebooks and :opq for 'cartesian' k-means clustering codebooks

• distance::PreMetric the distance to be used in the

codebook generation methods and data encoding

Other codebook generation algorithm specific keyword arguments such as maxiter::Int can be specified as well.

quantize(aa; [;kwargs])

Quantize an array aa.

Arguments

• aa::AbstractArray input array to be quantized

Keyword arguments

• k::Int the number of vector prototypes in each codebook
• m::Int the number of codebooks
• method::Symbol the algorithm to be employed for codebook

generation; possible values are :sample (default), :pq for classical k-means clustering codebooks and :opq for 'cartesian' k-means clustering codebooks

• distance::PreMetric the distance to be used in the

codebook generation methods and data encoding

Other codebook generation algorithm specific keyword arguments such as maxiter::Int can be specified as well.

quantize(aq, aa)

Quantize an array aa using an array quantizer aq.

Abstract quantization type. Its subtypes are two singletons

corresponding to additive/residual and orthogonal quantization.

codebooks(aq)

Access the codebooks field of the array quantizer aq.

decode(codebook, codes)

Returns a partial array reconstruction for using the codes and codebook.

encode(codebook, aa [;distance=DEFAULT_DISTANCE])

Encodes the input array aa using distance to calculate the closest vector prototype from the codebook.

nvars(aa)

Returns the number of variables of an array aa. For vectors, the value is always 1, for matrices it is the number of rows in the matrix.

opq_codebooks(X, k, m [;distance=DEFAULT_DISTANCE, maxiter=DEFAULT_PQ_MAXITER])

Build m codebooks for input matrix X using k sub-space centers obtained using 'cartesian' k-means clustering, the distance distance and maxiter iterations.

References:

• Ge et al. 2014

pq_codebooks(X, k, m [;distance=DEFAULT_DISTANCE, maxiter=DEFAULT_PQ_MAXITER])

Build m codebooks for input matrix X using k sub-space centers obtained using k-means clustering, the distance distance and maxiter iterations.

References:

• Jègou et al. 2011

quantization_type(method)

Returns a quantization type based on the quantization method to be employed.

quantize_data(aq, aa)

Returns a quantized version of the array aa using the array quantizer aq.

quantized_eltype(k)

Determinies the minimum Unsigned type that can hold the value k.

quantizer(qa)

Access the quantizer field of a quantized array qa.

rowrange(n, m, i)

Utility function that returns a range based on an iteration index i, the number of elements n and number of ranges m.

rvq_codebooks(X, k, m [;distance=DEFAULT_DISTANCE, maxiter=DEFAULT_RVQ_MAXITER])

Build m codebooks for input matrix X using k layers of residual vector cluster centers, obtained using k-means clustering, the distance distance and maxiter iterations.

References:

• Chen et al. 2010

sampling_codebooks(X, k, m)

Build m codebooks for input matrix X by sampling k prototype vectors.