Efficient Optimization of Quantized Models

LOTION: Smoothing the Optimization Landscape for Quantized Training

A research team at the Kempner Institute developed LOTION, a principled framework for smoothing the discontinuous optimization landscape encountered in quantized neural network training by replacing the raw quantized loss with its expectation under randomized-rounding noise, enabling standard optimizers to converge reliably while preserving all global minima of the original problem.

Read the blog post