События
1754: A French mathematician D'Alembert uses
convolutions in a derivation of Taylor's Theorem.
This is one of the first known instances of the use
of convolutions.1951: (SGD) A precursor to gradient descent is
presented in the paper "A Stochastic
Approximation Method."
Other sources point to gradient descent being
used in control theory as far back as 1940 but I
couldn't find actual publications confirming this.1952: (SGD) "Stochastic Estimation of the
Maximum of a Regression Function" is
published. This paper contains a complete
description of stochastic gradient descent with a
batch size of 1.DNN
The Perceptron: A Probabilistic Model For
Information Storage And Organization In
The Brain
Introduction paper to the perceptron (single
layer of a DNN)1971: (DNN) First working multilayer perceptron
model introduced in Cybernetics and forecasting
techniques. Not trained with SGD.1970: (Backprop) "The representation of the
cumulative rounding error of an algorithm as a
Taylor expansion of the local rounding errors," a
masters thesis is presented by Seppo
Linnainmaa. Commonly said to be the first
presentation of backpropagation.1980: (CNN) Neocognitron: A Self-organizing
Neural Network Model for a Mechanism of
Pattern Recognition Unaffected by Shift
in Position is published. First paper to introduce
the convolutional layer, based on work related to
monkey and cat visual cortex. Not trained with
SGD1986: The term deep learning is first used in the
machine learning paper "Learning while
searching in constraint-satisfaction problems"1989: Backpropagation applied to handwritten
zip code recognition - Today's paper!
LeCun creates one of the first applied, useful
deep convolutional neural network.1998: LeNet is presented in "Gradient-based
learning applied to document recognition"2012: AlexNet is presented in "ImageNet
classification with deep convolutional neural
networks"ResNet is presented in "Deep Residual Learning
for Image Recognition"Периоды
1950-1970: Two doctors, Hubel and Wiesel show
that cat and monkey visual cortexes contain
neurons that individually respond to small
regions of the visual field
Comments