Volume 1, Issue 3
Hartley Spectral Pooling for Deep Learning

Hao Zhang & Jianwei Ma

CSIAM Trans. Appl. Math., 1 (2020), pp. 518-529.

Published online: 2020-09

Preview Full PDF 74 1633
Export citation
  • Abstract

In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly called pooling. Maximization and averaging over sliding windows ($max/average$ $pooling$), and plain downsampling in the form of strided convolution are popular pooling methods. Since the pooling is a lossy procedure, a motivation of our work is to design a new pooling approach for less lossy in the dimensionality reduction. Inspired by the spectral pooling proposed by Rippel et al. [1], we present the Hartley transform based spectral pooling method. The proposed spectral pooling avoids the use of complex arithmetic for frequency representation, in comparison with Fourier pooling. The new approach preserves more structure features for network's discriminability than max and average pooling. We empirically show the Hartley pooling gives rise to the convergence of training CNNs on MNIST and CIFAR-10 datasets.

  • Keywords

Hartley transform, spectral pooling, deep learning.

  • AMS Subject Headings

68T07

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{CSIAM-AM-1-518, author = {Hao Zhang , and Jianwei Ma , }, title = {Hartley Spectral Pooling for Deep Learning}, journal = {CSIAM Transactions on Applied Mathematics}, year = {2020}, volume = {1}, number = {3}, pages = {518--529}, abstract = {

In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly called pooling. Maximization and averaging over sliding windows ($max/average$ $pooling$), and plain downsampling in the form of strided convolution are popular pooling methods. Since the pooling is a lossy procedure, a motivation of our work is to design a new pooling approach for less lossy in the dimensionality reduction. Inspired by the spectral pooling proposed by Rippel et al. [1], we present the Hartley transform based spectral pooling method. The proposed spectral pooling avoids the use of complex arithmetic for frequency representation, in comparison with Fourier pooling. The new approach preserves more structure features for network's discriminability than max and average pooling. We empirically show the Hartley pooling gives rise to the convergence of training CNNs on MNIST and CIFAR-10 datasets.

}, issn = {2708-0579}, doi = {https://doi.org/10.4208/csiam-am.2020-0018}, url = {http://global-sci.org/intro/article_detail/csiam-am/18306.html} }
TY - JOUR T1 - Hartley Spectral Pooling for Deep Learning AU - Hao Zhang , AU - Jianwei Ma , JO - CSIAM Transactions on Applied Mathematics VL - 3 SP - 518 EP - 529 PY - 2020 DA - 2020/09 SN - 1 DO - http://doi.org/10.4208/csiam-am.2020-0018 UR - https://global-sci.org/intro/article_detail/csiam-am/18306.html KW - Hartley transform, spectral pooling, deep learning. AB -

In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly called pooling. Maximization and averaging over sliding windows ($max/average$ $pooling$), and plain downsampling in the form of strided convolution are popular pooling methods. Since the pooling is a lossy procedure, a motivation of our work is to design a new pooling approach for less lossy in the dimensionality reduction. Inspired by the spectral pooling proposed by Rippel et al. [1], we present the Hartley transform based spectral pooling method. The proposed spectral pooling avoids the use of complex arithmetic for frequency representation, in comparison with Fourier pooling. The new approach preserves more structure features for network's discriminability than max and average pooling. We empirically show the Hartley pooling gives rise to the convergence of training CNNs on MNIST and CIFAR-10 datasets.

Hao Zhang & Jianwei Ma. (2020). Hartley Spectral Pooling for Deep Learning. CSIAM Transactions on Applied Mathematics. 1 (3). 518-529. doi:10.4208/csiam-am.2020-0018
Copy to clipboard
The citation has been copied to your clipboard