Amanda Knox was convicted of murder and her reputation sullied around the world, in large part because of her facial expressions and demeanour. Her story reveals how our instincts about others can be dangerously superficial, writes Ian Leslie

In the days and weeks following the discovery of Meredith Kercher's body, Italian police found no physical evidence linking Amanda Knox to the murder. But then, they didn't need it: they could tell Knox was guilty just by looking at her. "We were able to establish guilt," said Edgardo Giobbi, the lead investigator, "by closely observing the suspect's psychological and behavioural reaction during the interrogation. We don't need to rely on other kinds of investigation." Giobbi said that his suspicions were first raised just hours after the murder, at the crime scene, when he watched Knox execute a provocative swivel of her hips as she put on a pair of shoe covers.

Little about Knox's behaviour during that time matched how the investigators imagined a wrongfully accused woman should conduct herself. She appeared too cool and calm, they said – and yet also, it seems, oddly libidinous. One policeman said she "smelled of sex", and investigators were particularly disturbed by a video that first appeared on YouTube, shortly after the investigation began, which showed Knox and Raffaele Sollecito in each other's arms outside the cottage in which Kercher was murdered, as the investigation proceeded inside.

In fact, the video is anything but sexy. Knox, looking wan and dazed, exchanges chaste kisses with Sollecito, who rubs her arm consolingly. But the police professed shock. "Knox and Sollecito would make faces, kiss each other, while there was the body of a friend in those conditions," tutted Monica Napoleoni, head of Perugia's murder squad. A detective said he complained to Knox when she sat on Sollecito's lap, describing her behaviour as "inappropriate". Knox later explained to Rolling Stone magazine, via an intermediary, that she had been pacing up and down when Sollecito pulled her on to his knees to comfort her. The only strange thing about this is that an explanation for simple physical affection became necessary.

The Italian police's overheated interpretation of Knox's behaviour was a particularly pungent manifestation of a universal trait, one that frequently leads criminal investigators and juries astray: overconfidence in our ability to read someone else's state of mind simply by looking at them. This is not a uniquely modern error, born of pop psychology books. Shakespeare was wary of it. In Macbeth, he has Duncan remark how hard it is "to find the mind's construction in the face". It's a warning that law enforcement officers often seem unable, or unwilling, to heed.

In 2008 a group of Norwegian researchers ran an experiment to better understand how police investigators come to a judgment about the credibility of rape claims. Sixty-nine investigators were played video-recorded versions of a rape victim's statement, with the role of victim played by an actress. The wording of the statement in each version was exactly the same, but the actress delivered it with varying degrees of emotion. The investigators, who prided themselves on their objectivity, turned out to be heavily influenced in their judgments by assumptions about the victim's demeanour: she was judged most credible when crying or showing despair.

In reality, rape victims react in the immediate aftermath of the event in a variety of ways: some are visibly upset; others are subdued and undemonstrative. There is, unsurprisingly, no universal reaction to being raped. The detectives were relying on their instincts, and their instincts turned out to be constructed from inherited and unreliable notions about women in distress.

Professional interrogators remain stubbornly convinced of their ability to tell if a person is being truthful simply by observing them. The lawyer and fraud expert Robert Hunter has termed this misapprehension "the demeanour assumption". He points out that it underpins the notions of oral evidence and jury trials; those who watch witnesses give evidence are assumed to be best placed to judge whether they are telling the truth.

It's not just police or legal professionals who make this error. We all have an inherent bias towards assuming that we can discern a person's inner mental state simply by observing them. Whether it's coining new theories on what the Mona Lisa was thinking, or wondering about the stranger opposite us on the tube, we possess an endless capacity to speculate on a person's character, thoughts, and motivations based on the slender evidence of a facial expression. The eyes, it is said, are windows to the soul. They are not. They are organs for converting light into electro-magnetic impulses. But this has never stopped us dreaming of them that way.

Amanda Knox wasn't able to communicate her thoughts and feelings directly, either to the police or to the wider public. Her Italian, at the time of the murder, was poor, and her arraignment meant that she couldn't speak to the media. But there were plenty of pictures to go on. There was, therefore, an even greater emphasis on her expressions and physical behaviour than there would normally be in such a situation, right from the beginning.

This focus on the superficial shaped not just Knox's fortunes in the original trial, but her reputation around the world. Italian prosecutors were quick to leak stories about Knox doing cartwheels while in custody, because they knew the image, even if only imagined, would lead people to conclude that she was guilty. When the press published pictures of Knox with a smile on her face, readers around the world reacted the same way: no innocent person accused of a crime would behave like this. An Italian friend of Kercher's, Giacomo Silenzi, was widely quoted: "Her eyes didn't seem to show any sadness, and I remember wondering if she could have been involved." The tape of Knox embracing Sollecito was played over and over, often with a commentary suggesting there was something odd or distasteful about it.

It is astonishing how quick we are to draw conclusions about how a person ought to look or behave in circumstances we haven't ourselves even come close to experiencing. How many of us have returned to our home after a night away to discover that our flatmate has been brutally murdered? How many of us can know what it feels like to be at the sharp end of a punishing interrogation, in a foreign country, carried out by men in uniform who seem absolutely convinced that they know what happened, who are as certain as we are confused, fearful and exhausted? None of us. And yet we feel free to blithely pronounce, from a great distance, on whether someone in this situation is "acting weird" or not.

What does it stem from, this over-confidence in facile intuitions about what other people are thinking? It probably has something to do with our innate difficulty in recognising that other people are as fully rounded and complex as we are. Emily Pronin, a psychologist at Princeton University, points out that there is a fundamental asymmetry about the way two human beings relate to one another in person. When you meet someone, there are at least two things more prominent in your mind than in theirs – your thoughts, and their face. As a result we tend to judge others on what we see, and ourselves by what we feel. Pronin calls this "the illusion of asymmetric insight".

You know when you're hiding your true thoughts and feelings – pretending to be fascinated by your boss's endless anecdote, or grinning your way through an agonising first date – but you nonetheless tend to assume the other's appearance tells the full story of how they feel: if she's smiling, it's because she's genuinely enjoying herself.

Studies have found that people over-estimate how much they can learn from others in job interviews, while at the same time maintaining that others can only get a glimpse of them from such brief encounters. The model we seem to work with is something like this: I am infinitely subtle, complex and never quite what I seem; you are predictable and straightforward, an open book.

Of course, the asymmetry is likely to be particularly lopsided when we don't know the person; when we have only seen pictures of them on TV, in newspapers and magazines. We know for sure that they don't know us, and yet we're almost equally certain that we've got their number. We only had to look at Knox's various expressions – in her Facebook and MySpace photos, and in pictures of her on the way to court – to make snap judgments about what kind of person she was, or what was running through her mind when the picture was taken. Even this week, photographs of her smiling as she boards a plane home have been interpreted as a sign of callousness.

An inclination to oversimplify the minds and motivations of others lies at the root of sexism and racism, and all forms of inter-group conflict, violent and benign. Liberals and conservatives tend to think that those on "their" side are reasonable, reflective, and thoughtful, while those on the other side are not just wrong, but simplistic and dim. Part of the reason that Knox became unpopular in Europe, and especially Italy, is that people projected on to her what they regarded as the worst qualities of Americans: arrogance, greed and brashness.

Our unwillingness to devote much effort to understanding how others might actually think or feel is exemplified by the popular assumption that Knox's initial admission to police that she had been present at the scene of the murder, and her false implication of the bartender, Diya "Patrick" Lumumba, revealed a guilty conscience. "She lied!" declared her critics, slamming the collective gavel in condemnation. But of course we know, empirically, that under the extreme duress of an intense interrogation, a terrified person will say almost anything the police want them to say. Quite apart from falsely implicating others, people will falsely implicate themselves.

The Innocence Project is an American organisation devoted to exonerating those wrongfully convicted of serious crimes, frequently murder, by using DNA evidence. Of the 250 people they have successfully exonerated, a quarter had confessed under interrogation (Knox has indicated an interest in working for the Innocence Project, now she is free). Because we find it hard to imagine that we might do the same, we assume that others wouldn't do it either: a confession is still regarded by lawyers as the nuclear weapon of evidence, the one thing that, even in the absence of physical evidence, can guarantee conviction.

Most us know, when we reflect rationally, that other people are as complex and difficult to read or predict as we are, and we do compensate for the natural imbalance in our encounters with others. The trouble is, we rarely compensate enough. Thinking about what others might be thinking and feeling is hard work. It requires intellectual application, empathy, and imagination. Most of the time we can barely be bothered to exert such efforts on behalf of our friends and partners, let alone on people we read about in the news. We fall back on guesses, stereotypes, and prejudices. This is inevitable, and not always a bad thing. The trouble comes when we confuse our short-cuts with judgment.

Amanda Knox's face proved to be her misfortune. It was pretty enough to incite the fantasies of Italian cops and tabloid editors, and just expressive enough to provide a richly textured canvas for a public all too willing to pronounce on the soul it concealed.

Ian Leslie is the author of Born Liars; Why We Can't Live Without Deceit. He is on Twitter @mrianleslie.