Learning binary data representation for optical processing units

Kozyrskiy, Bogdan; Filippone, Maurizio; Poli, Iacopo; Ohana, Ruben; Daudet Laurent; Carron, Igor
Sensors and Transducers, Vol.256, N°2, 31 March 2022

Optical Processing Units (OPUs) are computing devices that perform random projections of input data by exploiting the physical phenomenon of scattering a light source through a diffusive medium. Random projections calculated by OPUs have been used successfully for approximating kernel ridge regression for large datasets with low power consumption and at high speed. However, OPUs require the input data to be binary. In this paper, we propose to use shallow and deep neural networks (NN) as binary encoders to perform input data binarization. The difficulty in developing a binarization strategy which is learned in an end-to-end fashion along with kernel ridge regression parameters, is due to the non-differentiability of the operation performed by the OPU. We overcome this difficulty by considering OPUs as a black-box and by employing the REINFORCE gradient estimator, which allows us to calculate the gradient of the loss function with respect to the weights of the binarization encoder and to optimize these together with the parameters of kernel ridge regression with gradientbased optimization. Through our experimental campaign on a variety of tasks and datasets, we show that our method outperforms alternative unsupervised and supervised binarization techniques. 


Type:
Journal
Date:
2022-03-31
Department:
Data Science
Eurecom Ref:
6892
Copyright:
IFSA Publishing
See also:

PERMALINK : https://www.eurecom.fr/publication/6892