Once it gets off the ground into space all science fiction is fantasy, and the more naturalistic the traditional form tries to be, the greater its failure. The new science fiction explores the point where outer reality and inner psyche meet

During the past few years it has become apparent that science fiction, long regarded as a self-contained mélange of post-Wellsian fantasies about time and space, is dividing itself into two separate and opposed forms. On the one hand is the traditional science fiction of interplanetary travel and alien cultures, in its heyday thirty years ago an authentic popular image of the future, but now identified with a few declining magazines and a group of older American and British writers unable to add anything substantially new to their repertory of ideas.

Their failure, and that of the entire genre, in spite of the heroic efforts of intelligent apologists such as Kingsley Amis and Edmund Crispin, not to mention the millions of dollars' worth of free publicity given by the Russo-American space programmes, lies in its mistaken appeal to realism rather than to fantasy. Once it gets off the ground into space all science fiction is fantasy, and the more serious it tries to be, the more naturalistic, the greater its failure, since it completely lacks the moral authority and conviction of a literature won from experience.

On the other hand has emerged a more speculative form of science fiction, one that is crossing the horizon of general fiction at an increasing number of points. Where the older science fiction has been most involved with outer space, this new offshoot is concerned with "inner space," the surrealists' "landscapes of the soul," and in creating images where the outer world of reality and the inner world of the psyche meet and fuse. Indeed, for those writers science serves much the same role as did psychoanalysis for the surrealists - a standpoint rather than a subject matter.

Recognising that the whole of science fiction's imaginary universe has long since been absorbed into the general consciousness and that most of its ideas are valid only in a kind of marginal spoofing, as in William Burroughs's "The Ticket that Exploded," they have set about trying to create a new set of conventions with which to explore their subject matter. As distinct from the teleological ends of science fiction in the past, with its explicit social and moral preoccupations, the new science fiction is devoted to ontological objectives - the understanding of time, landscape, and identity.

Facebook Twitter Pinterest JG Ballard at his typewriter, March 1989. Photograph: David Montgomery/Getty Images

This distinction between outer and inner space is apparent in the latest Kingsley Amis and Robert Conquest collection Spectrum IV (Gollancz, 21s). As usual, the sharpness of the editorial eye produces many stories that convince by sheer expertise, but the best are those, such as the late C. M. Kornbluth's "The Marching Morons" and John Brunner's "Such Stuff," which are extrapolations of the immediate present, nightmares at noon earned from the abrasive dust of the pavements we all walk.

Telepathy, one of the most hazardous subjects, succeeds in the skilled hands of John Brunner in Telepathist (Faber, 18s). The full implications of telepathy require nothing less than genius to do them justice, but in his tale of a deformed and neglected child who meets a deaf-mute he convinces us that some kind of extrasensory perception might be the only spark to cross the gap between them. Intelligently and humanely told, the novel succeeds more by the use of traditional literary devices than those of science fiction.

An Arthur C Clarke Omnibus (Sidgwick and Jackson, 30s) contains two novels, "Childhood's End" and "Prelude to Space," and a short-story collection, "Expedition to Earth." Reprinted after a ten-year lapse, they illustrate the failure of traditional science fiction. Wholly concerned with an outer space seen in terms of the crudest extrapolations, these stories are dated not only by their superficial scientific gimmickry, but by the trivial dialogue and characterisation. The difference between the old and new science fiction is the point where invention ends and imagination begins.