TY - JOUR T1 - A Brief Survey on the Approximation Theory for Sequence Modelling AU - Jiang , Haotian AU - Li , Qianxiao AU - Li , Zhong AU - Wang , Shida JO - Journal of Machine Learning VL - 1 SP - 1 EP - 30 PY - 2023 DA - 2023/03 SN - 2 DO - http://doi.org/10.4208/jml.221221 UR - https://global-sci.org/intro/article_detail/jml/21511.html KW - Approximation theory, Sequence modelling, Machine learning, Deep learning, Dynamics. AB -

We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various model architectures through the lens of classical approximation paradigms, and the insights one can gain from these results. We also outline some future research directions towards building a theory of sequence modelling.