为了方便重现我们ECML2015论文中提出的高速的PROXTONE优化方法，在这里公开我们的优化代码。PROXTONE可以用于优化两个凸函数的和：其中一是大量光滑凸函数的平均，另一是作为惩罚项或正则项的非光滑凸函数。PROXTONE通过结合目标函数的二阶信息（secondorder information）获得了更强收敛结果。在此类问题中，数值试验表明目前PROXTONE获得了最快的速度。PROXTONE方法可以用于稀疏的逻辑回归，稀疏的深度学习等。

PROXTONE is a function optimizerfor the case where the target function is the sums of two convex functions: oneis the average of a huge number of smooth convex functions, and the other is anonsmooth convex function. It incorporates second order information to obtain stronger convergence results. It does not require tuning of hyperparameters. Itis described in more detail in the ECML2015 paper:

@incollection{shi2015large,

 title={Large ScaleOptimization with Proximal Stochastic Newton-Type Gradient Descent},

 author={Shi, Ziqiang andLiu, Rujie},

 booktitle={MachineLearning and Knowledge Discovery in Databases},

 pages={691--704},

 year={2015},

 publisher={Springer}

}

This blog provides easy to use MATLAB implementations ofPROXTONE, as well as functions to reproduce the research in the paper.

Use PROXTONE

The release code can be downloaded from http://vdisk.weibo.com/lc/2wZwi7oSEkT9nKIJ2r9(password：D239).

The example code which trains L1 regularized (sparse) logistic regressionis in proxtone_lr.m. To use PROXTONE you just call ‘proxtone_lr’ in thecommand window of matlab.

The following figuresare painted by the script ‘proxtone_lr.m’. It can be seen that PROXTONE is thebest and parameter free.

Email shiziqiang@cn.fujitsu.com or shiziqiang7@163.com with questions if you don't find youranswer here.