When we witness the passage of time, according to noted physicist Paul Davies, we are actually observing how the “later states of the world differ from earlier states that we still remember.” In that sense time is like a movie: We are seeing slightly altered images playing in rapid succession.

Related Content Why Time is a Social Construct

That might explain why we perceive time as moving forward, but it fails to account for why we often perceive time moving at varying speeds. Why does it speed up when we’re having fun, but slow down when we’re bored? Most of us are fairly good at measuring short time intervals—seconds, minutes—but neuroscientists aren’t sure how we do it. “There is no one area of the brain, or system in the brain, that is solely dedicated to recording the passage of time,” says Marc Wittmann, a researcher at the Institute for Frontier Areas of Psychology and Mental Health, in Freiburg, Germany.

A leading theory is that we are endowed with the biological equivalent of a stopwatch: Somehow the brain emits a steady stream of pulses and subconsciously tallies how many were produced during a specific interval—maybe 100 pulses in your brain equals one minute. Recent research suggests that our brain runs multiple stopwatches simultaneously, depending on the kinds of tasks we’re engaged in. Unfortunately, it’s not clear what these pulses are, how they’re produced or where they’re tallied in the brain.

Scientists believe that our biological stopwatch makes use of several brain mechanisms, including those that control memory, regulate metabolism and process sensory inputs. Perhaps because those systems are themselves not rigidly fixed but sensitive to stimuli, the number of “timing” pulses that our brains emit per minute can increase or decrease, depending on our health, mood and even surroundings. For instance, evidence indicates that dopamine—a chemical neurotransmitter that helps regulate movement and emotion—affects our internal stopwatch. Patients who suffer from Parkinson’s disease have low dopamine levels and tend to underestimate the passage of time.

Emotional responses triggered by what we see or hear also affect our time perception. In a recent experiment, Wittmann and his colleagues had people stare at disks that appeared briefly on a computer screen. Sometimes the disks were static; other times they appeared to be coming closer or moving away. The subjects consistently overestimated the length of time that the “approaching” disks were on the screen. The reason? Wittmann speculates the oncoming disks were perceived as a potential threat—an evolutionary vestige that made us move or duck if something was approaching. So, we tense up. “Because of the threatening situation, I am totally aroused, and my internal physiological processes speed up,” says Wittmann. “And so relative to that, the outside world slows down.”

And what about that adage about time flying when we’re having fun? Wittmann says the answer has to do with attention. When we are focused on something we enjoy, our brain doesn’t keep track of the pulses emitted by our mental stopwatch. “You watch a thrilling movie, and 90 minutes pass like nothing,” says Wittmann, “but you’re waiting for the dentist for 90 minutes, and it just totally drags.”