Ghosts sightings have been making headlines recently. The Daily Star has dedicated several front pages to the supposed sighting of "black-eyed ghost children" over the past week; ITV reported that a couple said they had spotted the Grey Lady of Dudley; and the Metro reported a woman's claims that a ghost was writing letters "using her hands". It may be because Halloween is approaching, but it seems we remain fascinated by the idea of whether ghosts actually exist. The Guardian has gathered writers from the fields of psychology to religion to think about why.

Giles Fraser: 'There is something here that won't go away'

It may be an apocryphal story, but anthropologists often speak of it. One of those early-20th-century English anthropologist travels to Africa to expose the foolishness of what they presume to be "primitive" beliefs. "Do you believe ghosts exist?" he asks the man in the village. The man thinks, carefully. "I don't know if I believe in them," he replies, "but I am scared of them." Wise words.

Perhaps the most persistent attempt to expose the foolishness of the belief in ghosts is Scooby-Doo. Pretty much every week since 1969, when the series started, Scooby, Shaggy, Velma and the gang have been exposing the lie behind their fears of ghosts and ghoulies. It always turns out to be the strange caretaker with a clever projector, or some such; Velma's scientific turn of mind shows that Shaggy's fears are unfounded.

But here's my question: why do Shaggy and Scooby keep getting frightened, again and again? Are they just plain stupid? Or is there something about Velma's style of explanation that doesn't fully address the continual anxiety on which the series is premised? Despite the exposure of the caretaker, something remains under-explained – hence the continuously ongoing work of "Mystery, Inc".

From Pliny the Younger's story of an old man in chains haunting his house, through the stories of the Dybbuk, to the great gothic storytellers and the Blair Witch Project, in all cultures and times there is something here that won't go away; some fear that is legitimately being expressed – the continual return of the repressed. And the simple point that ghosts don't exist (obviously they don't, by the way) doesn't cut it. In a way, Velma is making a category mistake. The truth of ghosts is the way they represent our real and legitimate fears. And to this extent, ghosts are very, very real.

Giles Fraser is priest-in-charge at St Mary's Newington in south London, and a Guardian writer

María del Pilar Blanco: 'Ghosts endure in our cultural imagination'

Virtually all cultures have traditions of ghosts and haunting. Ghosts are inseparable from memory, history and loss. They represent how individuals and groups internalise their history, and how we reluctantly hold on to unshakeable past events. Ghosts can be site-specific: any given place can harbour multiple rumours and tales of sightings, possessions or the uncanny feeling of being "not alone".

One example is Chile's Atacama Desert, breathtakingly represented in Patricio Guzmán's documentary Nostalgia for the Light, at one time the site of Pinochet's concentration camps and where the relatives of the disappeared now search for remains that can allow them to give up their ghosts. Ghosts also keep up with our times and technologies. The ghost of Prudencio Aguilar in Gabriel García Márquez's One Hundred Years of Solitude, killed as a young man, ages alongside the Buendía family. Beyond the world of fiction, when science turned more ubiquitous in the 19th century Americans and Europeans became increasingly interested in proving the existence of ghosts. Spiritualism and other doctrines like theosophy spread from the US and Europe to the far corners of the world, reaching Cuba, Argentina and India.

Ghosts have acclimatised themselves to our media (think of the fictitious haunted videotapes in the Japanese horror film Ringu), and our new technologies look for them (think of the gadgets used by the numerous international ghost-hunting societies). While they may be linked to the past, ghosts endure in and are renovated by the cultural imagination of the present.

María del Pilar Blanco is a lecturer in Spanish American literature at the University of Oxford

Kira Cochrane: 'Who knows what accounts for these apparitions?'

Guardian Open Weekend: Kira Cochrane

I haven't seen a ghost myself, but my mother – an eminently practical, straightforward and unsuperstitious woman – once did. It was 1980, and my father had died suddenly of a heart attack eight months earlier, outside a sports centre after a game of squash, aged 34. This had left her with two small children (my brother Gleave was four, and I was two), and three months pregnant with my brother Frazer.

On Thursday she went to the hospital and was told my father had died; on Friday she was back to speak to the coroner, and decided to keep her first antenatal appointment, scheduled for that day. One shocked relative had suggested she should have an abortion, that she wouldn't be able to cope with three kids, but her doctor asked if she had been planning to have the baby before what had happened, and when she said she had he advised she continue as before.

My father had travelled a lot for work, and in those first months she says it was hard to believe what had happened, that he wasn't going to walk through the door again. Then, a few months after Frazer was born, she was breastfeeding him during the night and looked up to see my father standing at the end of the bed. "I came to see him," he said, gesturing to his son and namesake.

He was there just a moment, and my mother was unperturbed – as she says, when you talk to people openly and honestly about ghosts, many have had similar experiences. Who knows what accounts for these apparitions; are they an emanation of longing, love, hope, need? When close friends or relatives have died suddenly, I have sometimes looked at the retreating back of someone in a crowd whose shoulders hunch similarly, or who flicks their hair as the deceased once did – the thought that it might be them rushing and passing in a second's optimism. Perhaps we see ghosts because they help us to adjust, a hand reaching out to administer to the sudden, appalling wrench.

Kira Cochrane is a Guardian writer

Christopher French: 'Misinterpretation of natural phenomena'

There are many reasons to be extremely sceptical regarding the existence of ghosts from a psychological perspective. For one thing, the idea that some kind of consciousness could survive bodily death is completely at odds with modern neuroscience. For another, if spirits of the dead really did survive in some form, we might expect their appearance and behaviour to reflect some kind of eternal unchanging afterlife. In fact, accounts of ghosts from different cultures vary considerably. So what might explain the fact that so many people believe in ghosts, and a sizeable minority claim they have personally encountered a ghost?

A minority of cases, such as the infamous Amityville Horror, are based upon deliberate hoaxes, but most claims are undoubtedly sincere. They may arise as the result of honest misinterpretation of naturally occurring phenomena (for example, seeing faces and figures in the shadows or hearing noises made by animals in the night), or common but frightening anomalous experiences. An example of the latter would be sleep paralysis episodes that can occur between sleep and wakefulness. During such episodes, the sufferer is temporarily paralysed and may experience a strong sense of presence and a variety of bizarre hallucinations. Although it can be terrifying, sleep paralysis is essentially harmless.

Women are much more likely to believe in ghosts than men, and to report that they have personally experienced a ghost (in line with most other paranormal phenomena), but there are no clear trends relating to age. Not surprisingly perhaps, fantasy-prone personalities are much more likely to report having encountered a ghost.

Our fear of our own mortality plays an important role in belief in ghosts. Most of us desperately want to believe in life after death – and the idea of ghosts, however scary, seems to offer support for such a notion.

Christopher French is head of the Anomalistic Psychology Research Unit at Goldsmiths, University of London