1. Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

2. Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B. & Uchida, N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88 (2012).

3. Roitman, M. F., Wheeler, R. A., Wightman, R. M. & Carelli, R. M. Real-time chemical responses in the nucleus accumbens differentiate rewarding and aversive stimuli. Nat. Neurosci. 11, 1376–1377 (2008).

4. Matsumoto, H., Tian, J., Uchida, N. & Watabe-Uchida, M. Midbrain dopamine neurons signal aversion in a reward-context-dependent manner. eLife 5, e17328 (2016).

5. Tsai, H.-C. et al. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324, 1080–1084 (2009).

6. Ilango, A. et al. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 34, 817–822 (2014).

7. Chang, C. Y. et al. Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors. Nat. Neurosci. 19, 111–116 (2016).

8. Bayer, H. M. & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

9. Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841 (2009).

10. Menegas, W., Babayan, B. M., Uchida, N. & Watabe-Uchida, M. Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. eLife 6, e21886 (2017).

11. Kakade, S. & Dayan, P. Dopamine: generalization and bonuses. Neural Netw. 15, 549–559 (2002).

12. Lerner, T. N. et al. Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162, 635–647 (2015).

13. Parker, N. F. et al. Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target. Nat. Neurosci. 19, 845–854 (2016).

14. Howe, M. W. & Dombeck, D. A. Rapid signalling in distinct dopaminergic axons during locomotion and reward. Nature 535, 505–510 (2016).

15. Eshel, N. et al. Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525, 243–246 (2015).

16. Bonhomme, N., De Deurwaèrdere, P., Le Moal, M. & Spampinato, U. Evidence for 5-HT4 receptor subtype involvement in the enhancement of striatal dopamine release induced by serotonin: a microdialysis study in the halothane-anesthetized rat. Neuropharmacology 34, 269–279 (1995).

17. Ciabatti, E., González-Rueda, A., Mariotti, L., Morgese, F. & Tripodi, M. Life-long genetic and functional access to neural circuits using self-inactivating rabies virus. Cell 170, 382–392.e14 (2017).

18. Menegas, W., Uchida, N. & Watabe-Uchida, M. A self-killing rabies virus that leaves a trace on the DNA. Trends Neurosci. 40, 589–591 (2017).

19. Bar-Hillel, M. & Wagenaar, W. A. The perception of randomness. Adv. Appl. Math. 12, 428–454 (1991).

20. Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behaviour during a competitive game. Neural Netw. 19, 1075–1090 (2006).

21. LeDoux, J. & Daw, N. D. Surviving threats: neural circuit and computational implications of a new taxonomy of defensive behaviour. Nat. Rev. Neurosci. 19, 269–282 (2018).

22. Berlyne, D. E. Curiosity and exploration. Science 153, 25–33 (1966).

23. Moser, E. I., Moser, M. B. & Andersen, P. Potentiation of dentate synapses initiated by exploratory learning in rats: dissociation from brain temperature, motor activity, and arousal. Learn. Mem. 1, 55–73 (1994).

24. Albasser, M. M. et al. Perirhinal cortex lesions uncover subsidiary systems in the rat for the detection of novel and familiar objects. Eur. J. Neurosci. 34, 331–342 (2011).

25. James, W. The Principles of Psychology. (Holt, New York, NY, USA, 1890).

26. Tong, Q. et al. Synaptic glutamate release by ventromedial hypothalamic neurons is part of the neurocircuitry that prevents hypoglycemia. Cell Metab. 5, 383–393 (2007).

27. Yamaguchi, T., Qi, J., Wang, H.-L., Zhang, S. & Morales, M. Glutamatergic and dopaminergic neurons in the mouse ventral tegmental area. Eur. J. Neurosci. 41, 760–772 (2015).

28. Menegas, W. et al. Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass. eLife 4, e10032 (2015).

29. Fiorillo, C. D., Song, M. R. & Yun, S. R. Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J. Neurosci. 33, 4710–4725 (2013).

30. Ghazizadeh, A., Griggs, W. & Hikosaka, O. Ecological origins of object salience: reward, uncertainty, aversiveness, and novelty. Front. Neurosci. 10, 378 (2016).

31. Krusemark, E. A. & Li, W. Do all threats work the same way? Divergent effects of fear and disgust on sensory perception and attention. J. Neurosci. 31, 3429–3434 (2011).

32. Lee, D. H. & Anderson, A. K. Reading what the mind thinks from how the eye sees. Psychol. Sci. 28, 494–503 (2017).

33. Oleson, E. B., Gentry, R. N., Chioma, V. C. & Cheer, J. F. Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. J. Neurosci. 32, 14804–14808 (2012).

34. Rescorla, R. A. & Wagner, A. R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: A. H. Black, W. F. Prokasy eds.. Classical Conditioning II: Current Research and Theory (pp. 64–99. Appleton-Century-Crofts, New York, NY, USA, 1972).

35. Lloyd, K. & Dayan, P. Safety out of control: dopamine and defence. Behav. Brain Funct. 12, 15 (2016).

36. Rogan, M. T., Leon, K. S., Perez, D. L. & Kandel, E. R. Distinct neural signatures for safety and danger in the amygdala and striatum of the mouse. Neuron 46, 309–320 (2005).

37. Guarraci, F. A., Frohardt, R. J. & Kapp, B. S. Amygdaloid D1 dopamine receptor involvement in Pavlovian fear conditioning. Brain Res. 827, 28–40 (1999).

38. Kishioka, A. et al. A novel form of memory for auditory fear conditioning at a low-intensity unconditioned stimulus. PLoS One 4, e4157 (2009).

39. Paré, D. & Quirk, G. J. When scientific paradigms lead to tunnel vision: lessons from the study of fear. NPJ Sci. Learn. 2, 6 (2017).

40. Kita, H. & Kitai, S. T. Amygdaloid projections to the frontal cortex and the striatum in the rat. J. Comp. Neurol. 298, 40–49 (1990).

41. Loughlin, S. E. & Fallon, J. H. Dopaminergic and non-dopaminergic projections to amygdala from substantia nigra and ventral tegmental area. Brain Res. 262, 334–338 (1983).

42. Darvas, M. & Palmiter, R. D. Restriction of dopamine signaling to the dorsolateral striatum is sufficient for many cognitive behaviors. Proc. Natl. Acad. Sci. USA 106, 14664–14669 (2009).

43. Schiemann, J. et al. K-ATP channels in dopamine substantia nigra neurons control bursting and novelty-induced exploration. Nat. Neurosci. 15, 1272–1280 (2012).

44. Choi, J.-S. & Kim, J. J. Amygdala regulates risk of predation in rats foraging in a dynamic fear environment. Proc. Natl. Acad. Sci. USA 107, 21773–21777 (2010).

45. Ainsworth, M. D. & Bell, S. M. Attachment, exploration, and separation: illustrated by the behavior of one-year-olds in a strange situation. Child Dev. 41, 49–67 (1970).

46. Belin, D., Berson, N., Balado, E., Piazza, P. V. & Deroche-Gamonet, V. High-novelty-preference rats are predisposed to compulsive cocaine self-administration. Neuropsychopharmacology 36, 569–579 (2011).

47. Sasson, N. J., Turner-Brown, L. M., Holtzclaw, T. N., Lam, K. S. L. & Bodfish, J. W. Children with autism demonstrate circumscribed attention during passive viewing of complex social and nonsocial picture arrays. Autism Res. 1, 31–42 (2008).

48. Corey, D. T. The determinants of exploration and neophobia. Neurosci. Biobehav. Rev. 2, 235–253 (1978).

49. Waddell, S. Reinforcement signalling in Drosophila; dopamine does it all after all. Curr. Opin. Neurobiol. 23, 324–329 (2013).

50. Kim, H. F., Ghazizadeh, A. & Hikosaka, O. Dopamine neurons encoding long-term memory of object value for habitual behavior. Cell 163, 1165–1175 (2015).

51. Bäckman, C. M. et al. Characterization of a mouse strain expressing Cre recombinase from the 3′ untranslated region of the dopamine transporter locus. Genesis 44, 383–390 (2006).

52. Wickersham, I. R. et al. Monosynaptic restriction of transsynaptic tracing from single, genetically targeted neurons. Neuron 53, 639–647 (2007).

53. Chen, T.-W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).

54. Thiele, S.L., Warre, R. & Nash, J.E. Development of a unilaterally-lesioned 6-OHDA mouse model of Parkinson’s disease. J. Vis. Exp. https://doi.org/10.3791/3234 (2012).

55. Baldo, B. A., Daniel, R. A., Berridge, C. W. & Kelley, A. E. Overlapping distributions of orexin/hypocretin- and dopamine-β-hydroxylase immunoreactive fibers in rat brain regions mediating arousal, motivation, and stress. J. Comp. Neurol. 464, 220–237 (2003).

56. Gittis, A. H. et al. Rapid target-specific remodeling of fast-spiking inhibitory circuits after loss of dopamine. Neuron 71, 858–868 (2011).

57. Schallert, T., Fleming, S. M., Leasure, J. L., Tillerson, J. L. & Bland, S. T. CNS plasticity and assessment of forelimb sensorimotor outcome in unilateral rat models of stroke, cortical ablation, parkinsonism and spinal cord injury. Neuropharmacology 39, 777–787 (2000).

58. Athos, J. & Storm, D. R. High precision stereotaxic surgery in mice. Curr. Protoc. Neurosci. 4 Appendix, 4A (2001).

59. Uchida, N. & Mainen, Z. F. Speed and accuracy of olfactory discrimination in the rat. Nat. Neurosci. 6, 1224–1229 (2003).

60. Wang, A. Y., Miura, K. & Uchida, N. The dorsomedial striatum encodes net expected return, critical for energizing performance vigor. Nat. Neurosci. 16, 639–647 (2013).

61. Yizhar, O., Fenno, L. E., Davidson, T. J., Mogri, M. & Deisseroth, K. Optogenetics in neural systems. Neuron 71, 9–34 (2011).

62. Stujenske, J. M., Spellman, T. & Gordon, J. A. Modeling the spatiotemporal dynamics of light and heat propagation for in vivo optogenetics. Cell Rep. 12, 525–534 (2015).

63. Nakamura, K. & Hikosaka, O. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci. 26, 5360–5369 (2006).

64. Kudo, Y. et al. A single optical fiber fluorometric device for measurement of intracellular Ca2+concentration: its application to hippocampal neurons in vitro and in vivo. Neuroscience 50, 619–625 (1992).

65. Gunaydin, L. A. et al. Natural neural projection dynamics underlying social behavior. Cell 157, 1535–1551 (2014).

66. Kim, C. K. et al. Simultaneous fast measurement of circuit dynamics at multiple sites across the mammalian brain. Nat. Methods 13, 325–328 (2016).