Graduate School and Research Center in Digital Sciences

Kernel computations from large-scale random features obtained by optical processing units

Ohana, Ruben; Wacker, Jonas; Dong, Jonathan; Marmin, Sébastien; Krzakala, Florent; Filippone, Maurizio; Daude, Laurent

ICASSP 2020, IEEE International Conference on Acoustics, Speech and Signal Processing, 4-8 May 2020, Barcelona, Spain

Approximating kernel functions with random features (RFs) has been a successful application of random projections for nonparametric estimation. However, performing random projections presents computational challenges for large-scale problems. Recently, a new optical hardware called Optical Processing Unit (OPU) has been developed for fast and energy-efficient computation of large-scale RFs in the analog domain. More specifically, the OPU performs the multiplication of input vectors by a large random matrix with complexvalued i.i.d. Gaussian entries, followed by the application of an element-wise squared absolute value operation – this last nonlinearity being intrinsic to the sensing process. In this paper, we show that this operation results in a dot-product kernel that has connections to the polynomial kernel, and we extend this computation to arbitrary powers of the feature map. Experiments demonstrate that the OPU kernel and its RF approximation achieve competitive performance in applications using kernel ridge regression and transfer learning for image classification. Crucially, thanks to the use of the OPU, these results are obtained with time and energy savings.

Doi Arxiv Bibtex

Title:Kernel computations from large-scale random features obtained by optical processing units
Keywords:Kernel methods, nonparametric estimation, optical computing, random features, kernel ridge regression.
Type:Conference
Language:English
City:Barcelona
Country:SPAIN
Date:
Department:Data Science
Eurecom ref:6081
Copyright: © 2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Bibtex: @inproceedings{EURECOM+6081, doi = {http://dx.doi.org/10.1109/ICASSP40776.2020.9053272}, year = {2020}, title = {{K}ernel computations from large-scale random features obtained by optical processing units}, author = {{O}hana, {R}uben and {W}acker, {J}onas and {D}ong, {J}onathan and {M}armin, {S}{\'e}bastien and {K}rzakala, {F}lorent and {F}ilippone, {M}aurizio and {D}aude, {L}aurent}, booktitle = {{ICASSP} 2020, {IEEE} {I}nternational {C}onference on {A}coustics, {S}peech and {S}ignal {P}rocessing, 4-8 {M}ay 2020, {B}arcelona, {S}pain}, address = {{B}arcelona, {SPAIN}}, month = {05}, url = {http://www.eurecom.fr/publication/6081} }
See also: