On some similarities and differences between deep neural networks and kernel learning machines

Pei, Eddie; Fokoué, Ernest

On some similarities and differences between deep neural networks and kernel learning machines

Files

ma_11_1_pei_fokoue_final.pdf(1.77 MB)

Date

2022

Authors

Pei, Eddie

Fokoué, Ernest

Publisher

Vysoké učení technické v Brně, Fakulta strojního inženýrství, Ústav matematiky

Altmetrics

Abstract

This paper presents a thorough computational comparison of the predic- tive performances of deep neural networks and kernel learning machines. The work featured here successfully establishes that on both real-life datasets and artificially simulated ones, kernel learning machines tend to be just as good as deep neural net- works, and quite often outperform them predictively. It turns out from the findings of this paper that while deep neural networks might have worked well on tasks for which millions of observations are available, kernel learning machines just happen to be predictively better on a wide variety of tasks with the kind of sample size that one should realistically expect to have in practice.

Keywords

deep neural networks, kernel learning machines, single hidden layer neural net- works, support vector machine, cross validation

Citation

Mathematics for Applications. 2022 vol. 11, č. 1, s. 75-106. ISSN 1805-3629
http://ma.fme.vutbr.cz/archiv/11_1/ma_11_1_pei_fokoue_final.pdf

Document type

Peer-reviewed

Document version

Published version

Language of document

en

Document licence