Volume 1, Issue 4
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: What We Know and What We Don't

Weinan E, Chao Ma, Lei Wu & Stephan Wojtowytsch

CSIAM Trans. Appl. Math., 1 (2020), pp. 561-615.

Published online: 2020-12

Export citation
  • Abstract

The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also pay attention to the insight we have gained from careful numerical experiments as well as the analysis of simplified models. Along the way, we also list the open problems which we believe to be the most important topics for further study. This is not a complete overview over this quickly moving field, but we hope to provide a perspective which may be helpful especially to new researchers in the area.

  • Keywords

Neural networks, machine learning, supervised learning, regression problems, approximation, optimization, estimation, a priori estimates, Barron space, multi-layer space, flow-induced function space.

  • AMS Subject Headings

68T07, 26B40, 41A30, 35Q68

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{CSIAM-AM-1-561, author = {Weinan and E and and 10023 and and Weinan E and Chao and Ma and and 10024 and and Chao Ma and Lei and Wu and and 10025 and and Lei Wu and Stephan and Wojtowytsch and and 10026 and and Stephan Wojtowytsch}, title = {Towards a Mathematical Understanding of Neural Network-Based Machine Learning: What We Know and What We Don't}, journal = {CSIAM Transactions on Applied Mathematics}, year = {2020}, volume = {1}, number = {4}, pages = {561--615}, abstract = {

The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also pay attention to the insight we have gained from careful numerical experiments as well as the analysis of simplified models. Along the way, we also list the open problems which we believe to be the most important topics for further study. This is not a complete overview over this quickly moving field, but we hope to provide a perspective which may be helpful especially to new researchers in the area.

}, issn = {2708-0579}, doi = {https://doi.org/10.4208/csiam-am.SO-2020-0002}, url = {http://global-sci.org/intro/article_detail/csiam-am/18538.html} }
TY - JOUR T1 - Towards a Mathematical Understanding of Neural Network-Based Machine Learning: What We Know and What We Don't AU - E , Weinan AU - Ma , Chao AU - Wu , Lei AU - Wojtowytsch , Stephan JO - CSIAM Transactions on Applied Mathematics VL - 4 SP - 561 EP - 615 PY - 2020 DA - 2020/12 SN - 1 DO - http://doi.org/10.4208/csiam-am.SO-2020-0002 UR - https://global-sci.org/intro/article_detail/csiam-am/18538.html KW - Neural networks, machine learning, supervised learning, regression problems, approximation, optimization, estimation, a priori estimates, Barron space, multi-layer space, flow-induced function space. AB -

The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also pay attention to the insight we have gained from careful numerical experiments as well as the analysis of simplified models. Along the way, we also list the open problems which we believe to be the most important topics for further study. This is not a complete overview over this quickly moving field, but we hope to provide a perspective which may be helpful especially to new researchers in the area.

Weinan E, Chao Ma, Lei Wu & Stephan Wojtowytsch. (2020). Towards a Mathematical Understanding of Neural Network-Based Machine Learning: What We Know and What We Don't. CSIAM Transactions on Applied Mathematics. 1 (4). 561-615. doi:10.4208/csiam-am.SO-2020-0002
Copy to clipboard
The citation has been copied to your clipboard