Isaac Newton was a classic neurotic. He was a brooder and a worrier, prone to dwelling on the scientific problems before him as well as his childhood sins. But Newton also had creative breakthroughs–thoughts on physics so profound that they are still part of a standard science education.

In a Trends in Cognitive Sciences Opinion paper published August 27, psychologists present a new theory for why neurotic unhappiness and creativity go hand-in-hand. The authors argue that the part of the brain responsible for self-generated thought is highly active in neuroticism, which yields both of the trait’s positives (e.g., creativity) and negatives (e.g., misery).

People who score high on neuroticism in personality tests tend to have negative thoughts and feelings of all types, struggle to cope with dangerous jobs, and are more likely to experience psychiatric disorders within their lifetime. The most popular explanation for why people are neurotic comes from British psychologist Jeffrey Gray, who proposed in the 1970s that such individuals have a heightened sensitivity to threat. He reached this conclusion after observing how antianxiety drugs reduced the sensitivity of rodents to cues of punishment and also helped to relax and liven up psychiatric patients.

“Gray had a useful and logical theory, but the problem is that it doesn’t account for the full spectrum of neuroticism–it’s pretty difficult to explain neuroticism in terms of magnified threat perception because high scorers often feel unhappy in situations where there is no threat at all,” says paper lead author Adam Perkins, a personality researcher at King’s College London. “The second problem is, there’s literature showing neuroticism scores are positively correlated with creativity; and so why should having a magnified view of threat objects make you good at coming up with new ideas?”

Perkins’ eureka moment came after attending a lecture by coauthor and University of York psychologist Jonathan Smallwood, a leading expert on the neural basis of daydreaming. Smallwood described his research during the lecture, including a key study which showed that individuals at rest in an MRI scanner who spontaneously have particularly negative thoughts (a key marker of neuroticism) displayed greater activity in the regions of the medial prefrontal cortex that are associated with conscious perception of threat. Perkins realized that individual differences in the activity of these brain circuits that govern self-generated thought could be a causal explanation for neuroticism.

They collaborated with Dean Mobbs of the Columbia University Fear, Anxiety, and Biosocial Lab, who is an expert on the neural basis of defense in humans. Mobbs had previously shown that there is a switch from anxiety-related forebrain activity to panic-related midbrain activity as a threat stimulus moves closer. Mobbs had also showed that this switch from anxiety to panic is controlled by circuits in the basolateral nuclei of the amygdala–the brain’s emotional center.

“It occurred to me that if you happen to have a preponderance of negatively hued self-generated thoughts due to high levels of spontaneous activity in the parts of the medial prefrontal cortex that govern conscious perception of threat and you also have a tendency to switch to panic sooner than average people, due to possessing especially high reactivity in the basolateral nuclei of the amygdale, then that means you can experience intense negative emotions even when there’s no threat present,” Perkins says. “This could mean that for specific neural reasons, high scorers on neuroticism have a highly active imagination, which acts as a built-in threat generator.”

The psychiatric relevance of this theory was highlighted by psychiatrist and coauthor Danilo Arnone, who argued that this novel cognitive model might help to explain the ruminative thinking pattern seen in depression and is complementary to the already defined role of the subgenual prefrontal cortex in the aetiology of mood dysregulation.

The overthinking hypothesis also explains the positives of neuroticism. The creativity of Isaac Newton and other neurotics may simply be the result of their tendency to dwell on problems far longer than average people. “I keep the subject constantly before me, and wait till the first dawnings open slowly, by little and little, into a full and clear light,” Newton once said of his problem-solving method.

“We’re still a long way off from fully explaining neuroticism, and we’re not offering all of the answers, but we hope that our new theory will help people make sense of their own experiences, and show that although being highly neurotic is by definition unpleasant, it also has creative benefits,” Perkins says. “Hopefully our theory will also stimulate new research as it provides us with a straightforward unifying framework to tie together the creative aspects of neuroticism with its emotional aspects.”