Efficient Optimization of Quantized Models
LOTION: Smoothing the Optimization Landscape for Quantized Training
A research team at the Kempner Institute developed LOTION, a principled framework for smoothing the discontinuous optimization landscape encountered in quantized neural network training by replacing the raw quantized loss with its expectation under randomized-rounding noise, enabling standard optimizers to converge reliably while preserving all global minima of the original problem.