Regularization - Timeline

稀疏表示，特征选择（feature selection）机制。 Regression Shrinkage and Selection via the Lasso Robert Tibshirani 1996

作为约束的范数惩罚 Improving neural networks by preventing co-adaptation of feature detectors. Hinton, G. E 2012

作为约束的范数惩罚 Rank, trace-norm and max-norm. Srebro, N. 2005

数据集增强 (将噪声注入权重) An analysis of noise in recurrent neural networks: convergence and generalization. Jim, K.-C., Giles, C. L., and Horne, B. G. 1996

数据集增强 (将噪声注入权重) Practical variational inference for neural networks. Graves, A. 2011

半监督学习 Semi-Supervised Learning. Chapelle, O., Schölkopf, B., and Zien, A. 2006

数据集增强 (注入噪声) Turing computability with neural nets. Siegelmann, H. and Sontag, E. 1991

数据集增强 (注入噪声) Deep networks for robust visual recognition. Tang, Y. and Eliasmith, C. 2010

数据集增强 (注入噪声) Extracting and composingrobust features with denoising autoencoders. Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.-A. 2008

数据集增强 (注入噪声) Analyzing noise in autoencoders and deep networks. Poole, B., Sohl-Dickstein, J., and Ganguli, S. 2014

数据集增强 (注入噪声) Regularization and complexity control in feed-forward networks Bishop, C. M. 1995

数据集增强 (注入噪声) Training with noise is equivalent to Tikhonov regularization. Bishop, C. M. 1995

多任务学习 Multitask connectionist learning. Caruana, R. (1993).

多任务学习 Learning internal representations Baxter, J. (1995).

参数绑定和参数共享 Principled hybrids of generative and discriminative models. Lasserre, J. A., Bishop, C. M., and Minka, T. P. (2006).

Bagging 等集成方法 Bagging predictors. Breiman, L. (1994).

Dropout Dropout: A simple way to prevent neural networks from overfitting. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R 2014

生成对抗网络 Explaining and harnessing adversarial examples Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014b).

对抗网络（半监督学习方法） Distributional smoothing with virtual adversarial training Miyato, T., Maeda, S., Koyama, M., Nakae, K., and Ishii, S. (2015).

针对推导迁移学习的正则化使用论文里提出的 L2-SP 代替 L2 惩罚 A baseline regularization scheme for transfer learning with convolutional neural networks Xuhong Li, Yves Grandvalet, Franck Davoine 2019.9

半监督学习的 time span

提前终止（Early Stopping）这个方法一直在用