full
search.png

Search

close.png

Cancel

arrow
Volume 41, Issue 3
Extended Regularized Dual Averaging Methods for Stochastic Optimization

Jonathan W. Siegel & Jinchao Xu

DOI: 10.4208/jcm.2210-m2021-0106

J. Comp. Math., 41 (2023), pp. 525-541.

Published online: 2023-04

Preview Purchase PDF 894 11899

Cited by

google scholar semantic scholar
Export citation
  • Abstract

We introduce a new algorithm, extended regularized dual averaging (XRDA), for solving regularized stochastic optimization problems, which generalizes the regularized dual averaging (RDA) method. The main novelty of the method is that it allows a flexible control of the backward step size. For instance, the backward step size used in RDA grows without bound, while for XRDA the backward step size can be kept bounded. We demonstrate experimentally that additional control over the backward step size can speed up the convergence of the algorithm while preserving desired properties of the iterates, such as sparsity. Theoretically, we show that the XRDA method achieves the same convergence rate as RDA for general convex objectives.

  • Keywords

Convex Optimization, Subgradient Methods, Structured Optimization, Non-smooth Optimization.

  • AMS Subject Headings

90C25, 90C30

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address

jus1949@psu.edu (Jonathan W. Siegel)

jxx1@psu.edu (Jinchao Xu)

  • BibTex
  • RIS
  • TXT
@Article{JCM-41-525, author = {Siegel , Jonathan W. and Xu , Jinchao}, title = {Extended Regularized Dual Averaging Methods for Stochastic Optimization}, journal = {Journal of Computational Mathematics}, year = {2023}, volume = {41}, number = {3}, pages = {525--541}, abstract = {

We introduce a new algorithm, extended regularized dual averaging (XRDA), for solving regularized stochastic optimization problems, which generalizes the regularized dual averaging (RDA) method. The main novelty of the method is that it allows a flexible control of the backward step size. For instance, the backward step size used in RDA grows without bound, while for XRDA the backward step size can be kept bounded. We demonstrate experimentally that additional control over the backward step size can speed up the convergence of the algorithm while preserving desired properties of the iterates, such as sparsity. Theoretically, we show that the XRDA method achieves the same convergence rate as RDA for general convex objectives.

}, issn = {1991-7139}, doi = {https://doi.org/10.4208/jcm.2210-m2021-0106}, url = {http://global-sci.org/intro/article_detail/jcm/21637.html} }
TY - JOUR T1 - Extended Regularized Dual Averaging Methods for Stochastic Optimization AU - Siegel , Jonathan W. AU - Xu , Jinchao JO - Journal of Computational Mathematics VL - 3 SP - 525 EP - 541 PY - 2023 DA - 2023/04 SN - 41 DO - http://doi.org/10.4208/jcm.2210-m2021-0106 UR - https://global-sci.org/intro/article_detail/jcm/21637.html KW - Convex Optimization, Subgradient Methods, Structured Optimization, Non-smooth Optimization. AB -

We introduce a new algorithm, extended regularized dual averaging (XRDA), for solving regularized stochastic optimization problems, which generalizes the regularized dual averaging (RDA) method. The main novelty of the method is that it allows a flexible control of the backward step size. For instance, the backward step size used in RDA grows without bound, while for XRDA the backward step size can be kept bounded. We demonstrate experimentally that additional control over the backward step size can speed up the convergence of the algorithm while preserving desired properties of the iterates, such as sparsity. Theoretically, we show that the XRDA method achieves the same convergence rate as RDA for general convex objectives.

Jonathan W. Siegel & Jinchao Xu. (2023). Extended Regularized Dual Averaging Methods for Stochastic Optimization. Journal of Computational Mathematics. 41 (3). 525-541. doi:10.4208/jcm.2210-m2021-0106
Copy to clipboard
BibteX RIS TXT
The citation has been copied to your clipboard