After being initially heralded as having significant promise, the field of neural networks
has come in and out of fashion, particularly within the United States. From the 1980s through
the mid-2000s, their challenge seemed to be that there were significant limitations to the
technology that could not be easily fixed by using larger training datasets or through the
introduction of additional layers of “neurons.” However, in the mid-2000s, a small number of
new algorithmic approaches demonstrated the potential to enhance prediction through back
propagation through multiple layers. These neural networks increased their predictive power as
they were applied to larger and larger datasets, and were able to scale to an arbitrary level
(among others, a key reference here is Hinton and Salakhutdinov (2006)).