# larq.quantizers¶

A Quantizer defines the way of transforming a full precision input to a quantized output and the pseudo-gradient method used for the backwards pass.

Quantizers can either be used through quantizer arguments that are supported for Larq layers, such as input_quantizer and kernel_quantizer; or they can be used similar to activations, i.e. either through an Activation layer, or through the activation argument supported by all forward layer:

import tensorflow as tf
import larq as lq
...
x = lq.layers.QuantDense(64, activation=None)(x)
x = lq.layers.QuantDense(64, input_quantizer="ste_sign")(x)


is equivalent to:

x = lq.layers.QuantDense(64)(x)
x = tf.keras.layers.Activation("ste_sign")(x)
x = lq.layers.QuantDense(64)(x)


as well as:

x = lq.layers.QuantDense(64, activation="ste_sign")(x)
x = lq.layers.QuantDense(64)(x)


We highly recommend using the first of these formulations: for the other two formulations, intermediate layers - like batch normalization or average pooling - and shortcut connections may result in non-binary input to the convolutions.

## ste_sign¶

ste_sign(x)


Sign binarization function. $q(x) = \begin{cases} -1 & x < 0 \\ 1 & x \geq 0 \end{cases}$

The gradient is estimated using the Straight-Through Estimator (essentially the binarization is replaced by a clipped identity on the backward pass). $\frac{\partial q(x)}{\partial x} = \begin{cases} 1 & \left|x\right| \leq 1 \\ 0 & \left|x\right| > 1 \end{cases}$

Arguments

• x: Input tensor.

Returns

Binarized tensor.

References

## approx_sign¶

approx_sign(x)


Sign binarization function. $q(x) = \begin{cases} -1 & x < 0 \\ 1 & x \geq 0 \end{cases}$

The gradient is estimated using the ApproxSign method. $\frac{\partial q(x)}{\partial x} = \begin{cases} (2 - 2 \left|x\right|) & \left|x\right| \leq 1 \\ 0 & \left|x\right| > 1 \end{cases}$

Arguments

• x: Input tensor.

Returns

Binarized tensor.

References

## magnitude_aware_sign¶

magnitude_aware_sign(x)


Magnitude-aware sign for Bi-Real Net.

Arguments

• x: Input tensor

Returns

Scaled binarized tensor (with values in $\{-a, a\}$, where $a$ is a float).

References

## SteTern¶

SteTern(threshold_value: float = 0.05, ternary_weight_networks: bool = False) -> None

Instantiates a ternarization quantizer.

$q(x) = \begin{cases} +1 & x > \Delta \\ 0 & |x| < \Delta \\ -1 & x < - \Delta \end{cases}$

where $\Delta$ is defined as the threshold and can be passed as an argument, or can be calculated as per the Ternary Weight Networks original paper, such that

$\Delta = \frac{0.7}{n} \sum_{i=1}^{n} |W_i|$ where we assume that $W_i$ is generated from a normal distribution.

The gradient is estimated using the Straight-Through Estimator (essentially the Ternarization is replaced by a clipped identity on the backward pass). $\frac{\partial q(x)}{\partial x} = \begin{cases} 1 & \left|x\right| \leq 1 \\ 0 & \left|x\right| > 1 \end{cases}$

Arguments

• threshold_value: The value for the threshold, $\Delta$.
• ternary_weight_networks: Boolean of whether to use the Ternary Weight Networks threshold calculation.

Returns

Ternarization function

Aliases

• larq.quantizers.ste_tern

References