There's a reason your mother told you to look people in the eye when you talk to them: eye contact conveys important social cues. Yet when someone holds your gaze for more than a few seconds, the experience can take on a different tenor. New work elucidates the factors that affect whether we like or loathe locking eyes for a lengthy period.

Researchers have long known that eye contact is an important social signal. Our recognition of its import may even be hardwired. One study found that five-day-old babies prefer looking at faces that make direct eye contact compared with faces that have an averted gaze. “Eye contact provides some of the strongest information during a social interaction,” explains James Wirth, a social psychologist now at Ohio State University at Newark, because it conveys details about emotions and intentions. (Lack of eye contact is one of the early signs of autism in infants and toddlers.) The power of eye contact is so great that, according to a 2010 study co-authored by Wirth, if someone avoids your gaze for even a short period, you may feel ostracized.

But what determines how we feel about prolonged eye contact? One recent study explored this question. In research presented in May 2015 at the Vision Sciences Society conference, psychologist Alan Johnston and his colleagues at University College London collected information from more than 400 volunteers about their personalities. Then the subjects indicated their comfort level while watching video clips of actors who appeared to be looking directly at them for varying lengths of time.

Johnston and his colleagues found that, on average, the subjects liked the actors to make eye contact with them for 3.2 seconds, but the subjects were comfortable with a longer duration if they felt the actors looked trustworthy as opposed to threatening. “Gaze conveys that you are an object of interest, and interest is linked to intention,” Johnston explains—so if someone appears threatening and holds your gaze, that could indicate that the person has bad intentions. This idea could help explain findings from a controversial study published in 2013, which reported that people are more likely to change their views on a political issue when they are being challenged by people who do not make eye contact with them. If the challengers had made eye contact, they might have seemed more threatening and less trustworthy.

Our reaction to prolonged eye contact may relate to how we perceive ourselves, too. Johnston and his colleagues found that the more cooperative and warm subjects believed themselves to be, the longer they liked eye contact to be held. Johnston speculates that the more socially comfortable a person feels, the more he or she may “enjoy the intimacy of mutual gaze.”