Shame used to be an integral part of the justice system. From tarring and feathering to ducking stools, many societies have found it useful to turn the punishment of wrongdoing into a public spectacle. Halfway through his engrossing and terrifying So You’ve Been Publicly Shamed, Jon Ronson uncovers court records from 18th-century America and discovers, among the “various people named Nathaniel [who] had purchased land near rivers”, many accounts of people being whipped for adultery, or confined to the pillory for theft.

But he also finds something else, something altogether more alarming. “The common assumption is that public punishments died out in the new great metropolises because they’d been judged useless,” he writes. “But according to the documents I found that wasn’t it at all. They didn’t fizzle out because they were ineffective. They were stopped because they were far too brutal.” Public punishments fell out of fashion in the US and Britain from the 1830s: judges didn’t have a problem with whippings but they thought it was unnecessarily punitive to carry them out in public.

Now, Ronson says, we are seeing “a great renaissance of public shaming”. Social media, and Twitter in particular, has evolved into a rolling witch-hunt where reputations are smashed irreparably as the result of one ill-judged tweet or Facebook post. In extreme cases, a transgressor’s online identity never recovers: they bear the modern equivalent of being branded on the forehead.

The central incident in the book – what Ronson describes as the “brutal nadir” of internet shaming – is one I remember vividly. In December 2013, a PR executive called Justine Sacco boarded a plane at Heathrow heading for Cape Town, having tweeted from the runway: “Going to Africa. Hope I don’t get Aids. Just kidding. I’m white!”

Who was the butt of the joke: black sub-Saharan Africans, some of whom live in countries socially and economically devastated by a decades-long Aids epidemic? Or was Sacco satirising herself, a white first-worlder smugly imagining that race was a factor in the spread of the disease, rather than poverty or geography? It was hard to tell, not least because Twitter’s architecture – its 140-character limit – encourages abrupt, contextless thoughts to be ejected into the ether. Sacco probably thought that fewer than 170 people (the total number of her followers) would see the tweet, would swill it round in their minds, compare it with the Sacco they knew already, and understand that she wasn’t trying to be racist.

That is not what happened. The tweet was spotted by a Gawker journalist, Sam Biddle, who first retweeted it to his 15,000 followers and then wrote a blog post for the tech industry gossip site Valleywag that was seen by nearly 200,000 people. Most of them were angry. They found the tweet racist, offensive and hurtful. Soon, a hashtag – #hasjustinelandedyet – gained traction.

Now, deep breath. It is not my place as a white person to bang the gavel and tell people who have experienced a lifetime of racial discrimination that they were wrong to feel the joke was racist. But I am prepared to say this: the disproportionate nature of the reaction was demented.

What particularly horrified me was how the shaming played out in real time. Every­one following the saga on Twitter knew that Sacco was on a plane, without internet access, for 11 hours; soon, someone even worked out which flight she was on, based on the time and location stamp on her offending tweet. Self-righteous glee began to permeate the hashtag – Her life is ruined! She just doesn’t know it yet! – and I began to realise how intoxicating a public shaming could be. It delivers all the illicit thrill of sadism, overlaid with self-congratulatory mock-solemnity about how a Lesson Is Being Learned. No wonder Isabella and Ferdinand had no problem recruiting inquisitors, and crowds gather to watch Isis enforcers throw gay people off buildings. Who doesn’t want to do something bad while feeling good about themselves?

Tentatively, at the time, I tried to suggest that perhaps the tweet wasn’t that bad: certainly not bad enough to warrant the rape and death threats that were flooding in. Fellow tweeters began to argue that I was being a typical white, middle-class feminist, sticking up for a powerful PR executive and ignoring the voices of wronged people of colour. So I did something I have been ashamed of (the irony) ever since. I shut up and looked on as Justine Sacco’s life got torn apart. She lost her job, and is now known to millions as the “racist tweet lady”. She cannot date, for fear that someone will google her. Her aunt, a long-time supporter of the African National Congress, told her when she landed: “This is not what our family stands for.”

Lindsey Stone's Facebook photo.

The book has other examples of transgressors who have felt the brunt of an internet shaming. There is Jonah Lehrer, who lost his job at the New Yorker after confessing to plagiarism and fabricating quotations; Lindsey Stone, whose Facebook photo of her sticking up her middle finger in front of the sign at the veterans’ cemetery in Arlington went viral; and Max Mosley, who won damages from the News of the World for its allegation that he took part in a Nazi-themed BDSM orgy (it was a plain old German-themed BDSM orgy). Jon Ronson has made his name as a writer by adopting a naive, nebbish persona, and his specialism is catching awful people off-guard: in Them, his book about extremists, he politely listens to a Klansman explain how he will rehabilitate the white supremacist organisation with the help of advice garnered from slushy airport self-help books. Here, though, the awful people are us. Around his case studies, Ronson weaves historical and psychological research, including a reappraisal of the notorious Stanford prison experiment, to illustrate the cruel and random nature of crowd behaviour.

The big argument in favour of social media as a method of enforcing social norms is that it is more democratic than the old hierarchies. The internet is an attention economy, we are told – say something interesting or outrageous or inspiring enough, and people will listen. But what Ronson’s book shows is that mob rule can be profoundly conservative. He discovers that Mosley experienced few lasting negative effects from having his penchant for spanking covered by a tabloid, whereas many public shamings are “breathtakingly misogynistic”. The shamings of Sacco and Stone, which were accompanied by commentary about how they were “dumb bitches” who deserved everything they got as well as rape threats, are on a continuum with other methods of punishing transgressive women – the fad for “revenge porn”, the “slut-shaming” of teenage girls who are coerced into taking nude photos of themselves, or even websites dedicated to photographing “women eating on public transport”.

To me, public shaming is a symptom of institutional failure. It flourishes when people feel there is no accountability or possibility of redress through other channels. As our legal system matured and the concept of due process developed, we were content to outsource punishment to the state. Now, with trust in our institutions failing, we want to take matters into our own hands again. A year before Sacco’s tweet, an unarmed 17-year-old African-American called Trayvon Martin was shot dead by a neighbourhood watch volunteer who was later acquitted of all charges. Less than a year after the tweet, in the summer of 2014, an asthmatic 43-year-old called Eric Garner died telling an officer, “I can’t breathe,” as he was pinned to the floor in an unapproved chokehold; the following month, police units with military gear were sent in to Ferguson, Missouri, to quell unrest generated by the shooting of another black teenager, Michael Brown. (Neither man’s killer was put on trial.) Against this background of institutional racism in America, is it any wonder that so many tweeters felt unwilling to let what they perceived as a racist provocation pass without comment? One of Ronson’s interviewees says that “dragging down Justine Sacco felt like dragging down every rich white person who’s ever gotten away with making a racist joke because they could”.

This theme (that shaming flourishes when we have lost trust in the system) also runs through Jennifer Jacquet’s Is Shame Necessary?. While Ronson ends sombrely vowing never to take part in another public shaming, Jacquet sees shame as a weapon: powerful, but dangerous in the wrong hands. Her background is in environmentalism and she sees shaming as a way of encouraging good behaviour in firms that can’t be touched by legislators and regulators. If an energy giant is polluting the sea, or a fishing company killing dolphins, shaming them publicly bypasses politicians who might take campaign donations from them, or a legal system that it takes years – and thousands of dollars – to access.

Jacquet comes up with seven rules for who should be shamed, and how. These include the idea that victims of the transgression should be in charge of the shaming, and that there should be a big gap between desired and actual behaviour: she gives the example of direct-action groups of concerned citizens targeting bailed-out banks that give their executives huge bonuses. Shaming should be scrupulous and from a trusted source and should take place only where other methods are ineffective. But perhaps the most crucial point is this: “To keep shaming sustainable into the future, it is also important that individuals or companies who are exposed are given a chance to reintegrate within the group.” In the case of Greenpeace’s ethical seafood campaign, not only are wrongdoers called out, but stores at the top of the list are publicly congratulated, as are those that improve. This is what is missing from many social media shamings: any sense of what the transgressor can do to gain forgiveness or be rehabilitated. Justine Sacco, for instance, went to Ethiopia for several months to help with a maternal mortality prevention programme; but that will never have as much prominence on Google.

The other missing element is that old idea of “hate the sin, love the sinner”. It is comforting and easy to think that racism is perpetrated by a separate category of people – racists – who need to be identified and shunned. Yet we are all complicit in structures of discrimination and repression, and many largely good and well-meaning people act in ways that prop up oppressive systems. Hate the tweet, love the tweeter.

Just in case Ronson’s book doesn’t depress you enough about how new technology is reviving old uglinesses, turn to Andrew Keen’s The Internet Is Not the Answer. This brisk polemic synthesises not only Keen’s previous technosceptic thinking, but the work of almost every theorist who has had a downer on the internet in the past decade. There’s Jaron Lanier pointing out that mere access to social networks is pretty poor reward for all the commercially valuable data we give them; George Packer on how the hyper-libertarianism of Silicon Valley moguls shapes their creations; a dash of “superstar theory” (the idea that digital distribution creates a handful of mega-winners at the cost of the larger ranks of the merely very good); plus a sprinkle of the argument that automation hollows out middle-class jobs. Keen longs for the days when the internet was an academic project rather than a cash cow for three dozen venture capitalists in California. It’s completely one-sided, but all the more enjoyable for that.

You might, like me, finish reading these books and long to go and live in a yurt. Unfortunately, though, all of them are reminders that, whatever is wrong with the internet, it’s our fault. We can’t escape its flaws, because they are our own.