At the end of last winter, a gigantic billboard advertising Android, Google’s operating system, appeared over Times Square in New York. In a lower-case sans serif font – corporate code for friendly – it declared: “be together. not the same.” This erratically punctuated mantra sums up the web’s most magical proposition – its existence as a space in which no one need ever suffer the pang of loneliness, in which friendship, sex and love are never more than a click away, and difference is a source of glamour, not of shame.

As with the city itself, the promise of the internet is contact. It seems to offer an antidote to loneliness, trumping even the most utopian urban environment by enabling strangers to develop relationships along shared lines of interest, no matter how shy or isolated they might be in their own physical lives.

But proximity, as city dwellers know, does not necessarily mean intimacy. Access to other people is not by itself enough to dispel the gloom of internal isolation. Loneliness can be most acute in a crowd.

In 1942, the American painter Edward Hopper produced the signature image of urban loneliness. Nighthawks shows four people in a diner at night, cut off from the street outside by a curving glass window: a disquieting scene of disconnection and estrangement. In his art, Hopper was centrally concerned with how humans were handling the environment of the electric city: the way it crowded people together while enclosing them in increasingly small and exposing cells. His paintings establish an architecture of loneliness, reproducing the confining units of office blocks and studio apartments, in which unwitting exhibitionists reveal their private lives in cinematic stills, framed by panes of glass.

More than 70 years have passed since Nighthawks was painted, but its anxieties about connection have lost none of their relevance, though unease about the physical city has been superseded by fears over our new virtual public space, the internet. In the intervening years, we have entered into a world of screens that extends far beyond Hopper’s unsettled vision.

Loneliness centres on the act of being seen. When a person is lonely, they long to be witnessed, accepted, desired, at the same time as becoming intensely wary of exposure. According to research carried out over the past decade at the University of Chicago, the feeling of loneliness triggers what psychologists call hypervigilance for social threat. In this state, which is entered into unknowingly, the individual becomes hyperalert to rejection, growing increasingly inclined to perceive social interactions as tinged with hostility or scorn. The result is a vicious circle of withdrawal, in which the lonely person becomes increasingly suspicious, intensifying their sense of isolation.

Behind a computer screen, the lonely person has control. But the contact this produces is not the same thing as intimacy

This is where online engagement seems to exercise its special charm. Hidden behind a computer screen, the lonely person has control. They can search for company without the danger of being revealed or found wanting. They can reach out or they can hide; they can lurk and they can show themselves, safe from the humiliation of face-to-face rejection. The screen acts as a kind of protective membrane, a scrim that allows invisibility and transformation. You can filter your image, concealing unattractive elements, and you can emerge enhanced: an online avatar designed to attract likes. But now a problem arises, for the contact this produces is not the same thing as intimacy. Curating a perfected self might win followers or Facebook friends, but it will not necessarily cure loneliness, since the cure for loneliness is not being looked at, but being seen and accepted as a whole person – ugly, unhappy and awkward, as well as radiant and selfie-ready.

This aspect of digital existence is among the concerns of Sherry Turkle of the Massachusetts Institute of Technology, who has been writing about human-technology interactions for the past three decades. She has become increasingly wary of the capacity of online spaces to fulfil us in the ways we seem to want them to. According to Turkle, part of the problem with the internet is that it encourages self-invention. “At the screen,” she writes in Alone Together (2011), “you have a chance to write yourself into the person you want to be and to imagine others as you wish them to be, constructing them for your purposes. It’s a seductive but dangerous habit of mind.”

But there are other dangers. My own peak use of social media arose during a period of painful isolation. It was the autumn of 2011, and I was living in New York, recently heartbroken and thousands of miles from my family and friends. In many ways, the internet made me feel safe. I liked the contact I got from it: the conversations, the jokes, the accumulation of positive regard, the favouriting on Twitter and the Facebook likes, the little devices designed for boosting egos. Most of the time, it seemed that the exchange, the gifting back and forth of information and attention, was working well, especially on Twitter, with its knack for prompting conversation between strangers. It felt like a community, a joyful place; a lifeline, in fact, considering how cut off I otherwise was. But as the years went by – 1,000 tweets, 2,000 tweets, 17,400 tweets – I had the growing sense that the rules were changing, that it was becoming harder to achieve real connection, though as a source of information it remained unparalleled.

This period coincided with what felt like a profound shift in internet mores. In the past few years, two things have happened: a dramatic rise in online hostility, and a growing awareness that the lovely sense of privacy engendered by communicating via a computer is a catastrophic illusion. The pressure to appear perfect is greater than ever, while the once‑protective screen no longer reliably separates the domains of the real and the virtual. Increasingly, participants in online spaces have become aware that the unknown audience might at any moment turn on them in a frenzy of shaming and scapegoating.

The atmosphere of surveillance and punishment destroys intimacy by making it unsafe to reveal mistakes and imperfections. My own sense of ease on Twitter diminished rapidly when people began posting photos of strangers they had snapped on public transport, sleeping with their mouths open. Knowing that the internet was becoming a site of shaming eroded the feeling of safety that had once made it seem such a haven for the lonely.

The dissolution of the barrier between the public and the private, the sense of being surveilled and judged, extends far beyond human observers. We are also being watched by the very devices on which we make our broadcasts. As the artist and geographer Trevor Paglen recently said in the art magazine Frieze: “We are at the point (actually, probably long past) where the majority of the world’s images are made by machines for machines.” In this environment of enforced transparency, the equivalent of the Nighthawks diner, almost everything we do, from shopping in a supermarket to posting a photograph on Facebook, is mapped, and the gathered data used to predict, monetise, encourage or inhibit our future actions.

This growing entanglement of the corporate and social, this creeping sense of being tracked by invisible eyes, demands an increasing sophistication about what is said and where. The possibility of virulent judgment and rejection induces precisely the kind of hypervigilance and withdrawal that increases loneliness. With this has come the slowly dawning realisation that our digital traces will long outlive us.

Back in 1999, the critic Bruce Benderson published a landmark essay, Sex and Isolation, in which he observed: “We are very much alone. Nothing leaves a mark. Today’s texts and images may look like real carvings – but in the end they are erasable, only a temporary blockage of all-invasive light. No matter how long the words and pictures stay on our screens, there will be no encrustation; all will be reversible.”

Benderson thought the transience of the internet was the reason that it felt so lonely, but to me it is far more alarming to think that everything we do there is permanent. At that time – two years before 9/11, and 14 years before Edward Snowden exposed the intrusive surveillance it had set in motion – it was no doubt impossible to imagine the grim permanence of the web to come, where data has consequences and nothing is ever lost – not arrest logs, not embarrassing photos, not Google searches of child porn or embarrassing illnesses, not the torture records of entire nations.

Faced with the knowledge that nothing we say, no matter how trivial or silly, will ever be completely erased, we find it hard to take the risks that togetherness entails. But perhaps, as lonely people often are, I am being too negative, too paranoid. Perhaps we are capable of adapting, of finding intimacy in this landscape of unprecedented exposure. What I want to know is where we are headed. What is this sense of perpetual scrutiny doing to our ability to connect?

* * *

The future does not come from nowhere. Every new technology generates a surge of anxious energy. Each one changes the rules of communication and rearranges the social order. Take the telephone, that miraculous device for dissolving distance. From the moment in April 1877 that the first line linked phones No 1 and No 2 in the Bell Telephone Company, it was perceived as an almost uncanny instrument, separating the voice from the body.

The phone swiftly came to be regarded as a lifeline, an antidote to loneliness, particularly for rural women who were stuck in farmhouses miles from family and friends. But fears about anonymity clung to the device. By opening a channel between the outside world and the domestic sphere, the telephone facilitated bad behaviour. From the very beginning, obscene callers targeted both strangers and the “hello girls” who worked the switchboards. People worried that germs might be transmitted down the lines, carried on human breath. They also worried about who might be lurking, invisibly eavesdropping on private conversations. The germs were a fantasy, but the listeners were real enough, be they operators or neighbours on shared telephone lines.

Anxiety also collected around the possibility for misunderstanding. In 1930, Jean Cocteau wrote his haunting monologue The Human Voice, a play intimately concerned with the black holes that technologically mediated failures of communication produce. It consists of nothing more than a woman speaking on a bad party line – as these shared services were known – to the lover who has jilted her and who is imminently to marry another woman. Her terrible grief is exacerbated by the constant danger of being drowned out by other voices, or disconnected. “But I am speaking loud … Can you hear me? … Oh, I can hear you now. Yes, it was terrible, it was like being dead. You’re here and you can’t make yourself heard.” The final shot of the television film of the play, starring Ingrid Bergman, leaves no doubt as to the culprit, lingering grimly on the shining black handset, still emitting the dead end of a dial tone as the credits roll.

Photograph: Gail Albert-Halaban courtesy of Edwynn Houk Gallery

The broken, bitty dialogue of The Human Voice underscores the way that a device designed for talking might in fact make talking more difficult. If the telephone is a machine for sharing words, then the internet is a machine for constructing and sharing identities. In the internet era, Cocteau’s anxieties about how technology has affected our ability to speak intimately to one another accelerate into terror about whether the boundaries between people have been destroyed altogether.

I-Be Area, a chaotic, vibrant and alarming film made in 2007, turns on these questions of identity and its dissolution. Its central character is engaged in a war with his clone, and his clone’s online avatar. Making lavish use of jump cuts, face paint and cheap digital effects, the film captures the manic possibilities and perils of digital existence. All the cast, starting with the children in the first frame making hyper-cute adoption videos for themselves, are in search of a desirable persona. They perform for an audience that may at any moment dissolve or turn aggressive, which stimulates them into increasingly creative and bizarre transformations. Often seemingly imprisoned in teenage bedrooms, everyone is talking all the time: a tidal wave of rapid, high-pitched, Valley Girl inflections, the spiel of YouTube bedroom celebrities mashed with corporate catchphrases and the broken English of bots and programming lingo. Everyone is promoting, no one is listening.

The creator of this visionary and hilarious film is Ryan Trecartin, a baby-faced 34-year-old described by the New Yorker’s art critic, Peter Schjeldahl, as “the most consequential artist to have emerged since the 1980s”. Trecartin’s movies are made with a band of friends. They possess a campy DIY aesthetic that often recalls the avant garde genius of the 1960s film-maker Jack Smith, the character morphing of Cindy Sherman, the physical mayhem of Jackass and the idiotic confessional candour of reality TV.

These films take the experiences of contemporary digital culture – the sickening, thrilling feeling of being overwhelmed by a surge of possibilities, not least who you could become – and speed them up. Trecartin’s work is ecstatically enjoyable to watch, though as the critic Maggie Nelson wryly observes: “Viewers who look to Trecartin as the idiot savant emissary from the next generation who has come to answer the question ‘Are we going to be alright?’ are not likely to feel reassured.”

Watching the precisely crafted chaos, one has the disquieting sensation that it is one’s own life that is under the lens. Trecartin’s characters (though I doubt he would sanction such a term, with its vanished, 20th-century confidence in a solid knowable self) understand that they can be owned or branded, discarded or redesigned. In response to pressure, their identities warp and melt.

What is exciting about Trecartin’s work is the ecstasy generated by these transformations. It is tempting to suggest that this might even be a futuristic solution to loneliness: dissolving identity, erasing the burdensome, boundaried individual altogether. But there remain lingering currents of unease, not least around the question of who is watching.

* * *

Photograph: Gail Albert-Halaban courtesy of Edwynn Houk Gallery

For the past two years, Trecartin has been working with the curator Lauren Cornell to put together the 2015 Triennial at New York’s New Museum, which opened at the end of February. This event brings together 51 participants whose work reflects on internet existence. The title, Surround Audience, expresses the sinister as well as blissful possibilities for contact that have opened up. Artist as witness, or maybe artist imprisoned in an experiment none of us can escape.

Over the course of a freezing week in New York in February, I went to see Surround Audience four times, wanting to understand how contemporary artists were grappling with loneliness and intimacy. The most confrontationally dystopic piece was Josh Kline’s terrifying Freedom, an installation re-creating the architecture of Zuccotti Park, the privately owned public space in Manhattan that Occupy Wall Street took over. Kline had populated his replica with five human-size Teletubbies dressed in the uniforms of riot police, with thigh holsters, nine-hole boots and bulletproof vests. In their bellies were televisions playing footage of off-duty cops flatly read aloud from the social media feeds of activists. Kline’s work makes tangible the growing complication of the spaces we inhabit, and the easy misappropriation of our words. As I sat listening to the feed I watched a beaming young woman with a baby take repeated selfies with one of the helmeted figures.

What is it like to be watched like this? Many of the pieces suggest that it feels like being in prison – or perhaps in the horrifying quarantine bunkers designed by the Hong Kong artist Nadim Abbas. These tiny cells, no larger than a single bed, have been furnished, Apartment Therapy-style, with potted plants, striped throws and abstract prints, an atmosphere of modish domesticity at odds with the implicit violence of the space. As in Hopper’s Nighthawks diner, there is no way in or out; simply a pane of glass that facilitates voyeurism while making contact impossible. Touch can only be achieved by way of two sets of black rubber gauntlets, one pair permitting someone – a guard, maybe, or a nurse or warden – to reach in and the other allowing the incumbent to reach out. It’s hard to think of a lonelier space.

Frank Benson’s sculpture Juliana, 2015. Photograph: Benoit Pailley

But Surround Audience also includes work that testifies to the internet’s ability to dissolve isolation, to create community and closeness. Juliana, Frank Benson’s extraordinary sculpture of the 26-year-old artist and DJ Juliana Huxtable, is a triumphant icon of self-creation. Huxtable is transgender, and the sculpture, a life-size 3D print, displays her naked body, with both breasts and penis, those supposedly defining characteristics of gender. She reclines on a plinth, braids spilling down her back, her extended right hand fixed in a gesture of elegant command: a queenly figure, her shimmering skin spray-painted an unearthly metallic blue-green. Juliana shows how the trans community is redefining authenticity. It is not a coincidence that the trans rights movement has surged in an era in which both identity creation and community building are facilitated by technology. Turkle’s talk of the danger of self-creation misses the importance, especially for people whose sexuality, gender or race is considered marginal, of being able to construct and manifest an identity that is often off-limits or forbidden in the physical world.

* * *

The future does not announce its arrival. In Jennifer Egan’s Pulitzer prize-winning novel A Visit from the Goon Squad, published in 2010, there is a scene set in the near future that involves a business meeting between a young woman and an older man. After talking for a while, the girl becomes agitated by the demands of speech and asks the man if she can “T” him instead, though they are sitting side by side. As information silently flushes between their two handsets, she looks “almost sleepy with relief”, describing the exchange as pure. Reading it, I can distinctly remember thinking that it was appalling, shocking, wonderfully far-fetched. Within a matter of months it seemed instead merely plausible, a little gauche, but entirely understandable as an urge. Now it is just what we do: texting in company, emailing colleagues at the same desk, avoiding encounters, DMing instead.

While I was in New York, I met with Trecartin to discuss Surround Audience and what it has to say about the future we have fallen into. He was clutching a coffee and dressed in a red hoodie emblazoned with the word HUNT, a leftover prop from a shoot. He spoke much more slowly than the logorrheic characters he plays in films, pausing frequently to locate the exact word. He, too, felt that, with the acceleration in the past few years, we have entered almost unknowingly into a new era, long heralded and abruptly arrived. “We don’t necessarily look different yet, but we’re very different,” he said.

This space, the future now, is characterised, he believes, by a blurring between individuals and networks. “Your existence is shared and maintained and you don’t have control over all of it.”

But Trecartin feels broadly positive about where our embrace of technology might take us. “It’s obvious,” he said, “that none of this stuff can be controlled, so all we can do is steer and help encourage compassionate usage and hope things accumulate in ways that are good for people and not awful … Maybe I’m being naive about this, but all of these things feel natural. It’s like the way we already work. We’re making things that are already in us.”

We aren’t as solid as we once thought. We are embodied but also networks, living inside machines and people’s heads

The key word here is compassion, but I was also struck by his use of the word natural. Critiques of the technological society often seem possessed by a fear that what is happening is profoundly unnatural, that we are becoming post-human, entering what Turkle has called “the robotic moment”. But Surround Audience felt deeply human; an intensely life-affirming combination of curiosity, hopefulness and fear, full of richly creative strategies for engagement and subversion.

Over the week, I kept being drawn back by one piece in particular, an untitled six-minute film by the Austrian artist Oliver Laric, whose work is often about the tension between copies and originals. Laric has redrawn and animated scenes of physical transformation from dozens of cartoons, anchored by an odd, unsettlingly melancholy loop of music. Nothing stays constant. Forms continually migrate, a panther turning into a beautiful girl, Pinocchio into a donkey, an old woman deliquescing into mud. The people’s expressions are striking, as their bodies melt and reform, a heartrending mixture of alarm and resignation. The film captures our anxieties about image: Am I desirable? Do I need to be tweaked or improved? This sense of being out of control, subject to external and sinister forces, is part of what it has always meant to be human, to be trapped in temporal existence, with the inevitable upheavals and losses that entails. What could be more sci-fi, after all, than the everyday horror show of ageing, sickness, death?

Somehow, the vulnerability expressed by Laric’s film gave me a sense of hope. Talking to Trecartin, who is only three years younger than me, had felt like encountering someone from a different generation. My own understanding of loneliness relied on a belief in solid, separate selves that he saw as hopelessly outmoded. In his worldview, everyone was perpetually slipping into each other, passing through ceaseless cycles of transformation; no longer separate, but interspersed. Perhaps he was right. We aren’t as solid as we once thought. We are embodied but we are also networks, living on inside machines and in other people’s heads; memories and data streams. We are being watched and we do not have control. We long for contact and it makes us afraid. But as long as we are still capable of feeling and expressing vulnerability, intimacy stands a chance.

• Follow the Long Read on Twitter: @gdnlongread

Images by Gail Albert Halaban from the series Out My Window, showing until 2 May at Galerie Esther Woerdehoff, Paris, France.

Surround Audience is at the New Museum in New York until 24 May.

Olivia Laing’s book on loneliness, The Lonely City, will be published by Canongate in 2016.

