Geoffrey E. Hinton

Department of Computer Science

email: geoffrey [dot] hinton [at] gmail [dot] com University of Toronto voice: send email 6 King's College Rd. fax: scan and send email Toronto, Ontario

Information for prospective students:

I advise interns at Brain team Toronto.

I also advise some of the residents in the Google Brain Residents Program.

I will not be taking any more students, postdocs or visitors at the University of Toronto.



News

Results of the 2012 competition to recognize 1000 different types of object

How George Dahl won the competition to predict the activity of potential drugs

How Vlad Mnih won the competition to predict job salaries from job advertisements

How Laurens van der Maaten won the competition to visualize a dataset of potential drugs

Using big data to make people vote against their own interests

A possible motive for making people vote against their own interests





Basic papers on deep learning

LeCun, Y., Bengio, Y. and Hinton, G. E. (2015)

Deep Learning

Nature, Vol. 521, pp 436-444. [pdf]



Hinton, G. E., Osindero, S. and Teh, Y. (2006)

A fast learning algorithm for deep belief nets.

Neural Computation, 18, pp 1527-1554. [pdf]

Movies of the neural network generating and recognizing digits



Hinton, G. E. and Salakhutdinov, R. R. (2006)

Reducing the dimensionality of data with neural networks.

Science, Vol. 313. no. 5786, pp. 504 - 507, 28 July 2006.

[ full paper ] [ supporting online material (pdf) ] [ Matlab code ]

Papers on deep learning without much math



Hinton, G. E. (2007)

To recognize shapes, first learn to generate images

In P. Cisek, T. Drew and J. Kalaska (Eds.)

Computational Neuroscience: Theoretical Insights into Brain Function. Elsevier. [pdf of final draft]



Hinton, G. E. (2007)

Learning Multiple Layers of Representation.

Trends in Cognitive Sciences, Vol. 11, pp 428-434. [pdf]



Hinton, G. E. (2014)

Where do features come from?.

Cognitive Science, Vol. 38(6), pp 1078-1101. [pdf]



Recent Papers

Qin, Y., Frosst, N., Sabour, S., Raffel, C., Cottrell, C. and Hinton, G.

Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions

ICLR-2020 [pdf]



Kosiorek, A. R., Sabour, S., Teh, Y. W. and Hinton, G. E.

Stacked Capsule Autoencoders

Advances in Neural Information Processing Systems 32 [pdf]



Zhang, M., Lucas, J., Ba, J., and Hinton, G. E.

Lookahead Optimizer: k steps forward, 1 step back

Advances in Neural Information Processing Systems 32 [pdf]



Muller, R., Kornblith, S. and Hinton G. (2019)

When Does Label Smoothing Help?

Advances in Neural Information Processing Systems 32 [pdf]



Deng, B., Kornblith, S. and Hinton, G. (2019)

Cerberus: A multi-headed derenderer.

3D Scene Understanding Workshop, CVPR 2019 [pdf]



Deng, B., Genova, K., Yazdani, S., Bouaziz, S., Hinton, G. and Tagliasacchi, A. (2019)

Cvxnet: Learnable convex decomposition.

Perception as Generative Reasoning Workshop, NeurIPS 2019 [pdf]



Kornblith, S., Norouzi, M., Lee, H. and Hinton, G. (2019)

Similarity of neural network representations revisited

ICML-2019 [pdf]



Hinton, G. E., Sabour, S. and Frosst, N.

Matrix Capsules with EM Routing

ICLR-2018 [pdf]



Kiros, J. R., Chan, W. and Hinton, G. E.

Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search

ACL-2018 [pdf]



Anil, R., Pereyra, G., Passos, A., Ormandi, R., Dahl, G. and Hinton, G. E.

Large scale distributed neural network training through online distillation

ICLR-2018 [pdf]



Guan, M. Y., Gulshan, V., Dai, A. M. and Hinton, G. E.

Who Said What: Modeling Individual Labelers Improves Classification

AAAI-2018 [pdf]



Sabour, S., Frosst, N. and Hinton, G. E.

Dynamic Routing between Capsules

NIPS-2017, [pdf]



Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017)

Outrageously large neural networks: The sparsely-gated mixture-of-experts layer

arXiv preprint arXiv:1701.06538 [pdf]



Ba, J. L., Hinton, G. E., Mnih, V., Leibo, J. Z. and Ionescu, C. (2016)

Using Fast Weights to Attend to the Recent Past

NIPS-2016, arXiv preprint arXiv:1610.06258v2 [pdf]



Ba, J. L., Kiros, J. R. and Hinton, G. E. (2016)

Layer normalization

Deep Learning Symposium, NIPS-2016, arXiv preprint arXiv:1607.06450 [pdf]



Joseph Turian's map of 2500 English words produced by using t-SNE on the word feature vectors learned by Collobert & Weston, ICML 2008

Doing analogies by using vector algebra on word embeddings

My favorite Gary Marcus quote