Once in New York, I listened to a talk about the building of private prisons – a huge growth industry in the US. The prison industry needs to plan its future growth – how many cells are they going to need? How many prisoners are there going to be, 15 years from now? And they found they could predict the answers very easily, using a pretty simple algorithm, based on asking what percentage of 10 and 11-year-olds couldn't read. And certainly couldn't read for pleasure.

You can't say that a literate society has no criminality. But there are very real correlations. Literate people read fiction.

Fiction has two uses. First, it's a gateway drug to reading. The drive to know what happens next, to want to turn the page, the need to keep going, even if it's hard, because someone's in trouble and you have to know how it's all going to end … that's a very real drive. And it forces you to learn new words, to think new thoughts, to keep going. To discover that reading per se is pleasurable.

The simplest way to make sure that we raise literate children is to teach them to read, and to show them that reading is a pleasurable activity. And that means, at its simplest, finding books that they enjoy, giving them access to those books, and letting them read them. I don't think there is such a thing as a bad book for children. Every now and again it becomes fashionable among some adults to point at a subset of children's books, a genre, perhaps, or an author, and to declare them bad books, books that children should be stopped from reading. I've seen it happen over and over; Enid Blyton was declared a bad author, so was RL Stine, so were dozens of others. Comics have been decried as fostering illiteracy.

It's tosh. It's snobbery and it's foolishness. There are no bad authors for children, that children like and want to read and seek out, because every child is different. They can find the stories they need to, and they bring themselves to stories. A hackneyed, worn-out idea isn't hackneyed and worn out to them. This is the first time the child has encountered it. Do not discourage children from reading because you feel they are reading the wrong thing. Fiction you do not like is a route to other books you may prefer. And not everyone has the same taste as you.

Well-meaning adults can easily destroy a child's love of reading: stop them reading what they enjoy, or give them worthy-but-dull books that you like, the 21st-century equivalents of Victorian "improving" literature. You'll wind up with a generation convinced that reading is uncool and worse, unpleasant.

We need our children to get on to the reading ladder: anything they enjoy reading will move them up, rung by rung, into literacy. (Also, do not do what this author did when his 11-year-old daughter was into RL Stine, which is to go and get a copy of Stephen King's Carrie, saying, "If you liked those, you'll love this!" Holly read nothing but safe stories of settlers on prairies for the rest of her teenage years, and still glares at me when Stephen King's name is mentioned.)

The second thing fiction does is to build empathy. When you watch TV or see a film, you are looking at things happening to other people. Prose fiction is something you build up from 26 letters and a handful of punctuation marks, and you, and you alone, using your imagination, create a world and people it and look out through other eyes.

I was in China in 2007, at the first party-approved science fiction and fantasy convention in Chinese history. SF had been disapproved of for a long time. At one point I took a top official aside and asked him what had changed? "It's simple," he told me. "The Chinese were brilliant at making things if other people brought them the plans. But they did not innovate and they did not invent. They did not imagine. So they sent a delegation to the US, to Apple, to Microsoft, to Google, and they asked the people there who were inventing the future about themselves. And they found that all of them had read science fiction when they were boys or girls.

Fiction can show you a different world. It can take you somewhere you've never been. Once you've visited other worlds, like those who ate fairy fruit, you can never be entirely content with the world that you grew up in. Discontent is a good thing: discontented people can modify and improve their worlds, leave them better, leave them different.

And while we're on the subject, I'd like to say a few words about escapism. I hear the term bandied about as if it's a bad thing. As if "escapist" fiction is a cheap opiate used by the muddled and the foolish and the deluded, and the only fiction that is worthy, for adults or for children, is mimetic fiction, mirroring the worst of the world the reader finds herself in.

If you were trapped in an impossible situation, in an unpleasant place, with people who meant you ill, and someone offered you a temporary escape, why wouldn't you take it? And escapist fiction is just that: fiction that opens a door, shows the sunlight outside, gives you a place to go where you are in control, are with people you want to be with; and more importantly, during your escape, books can also give you knowledge about the world and your predicament, give you weapons, give you armour, real things you can take back into your prison. Skills and knowledge and tools you can use to escape for real.

Another way to destroy a child's love of reading, of course, is to make sure there are no books of any kind around. And to give them nowhere to read those books. I was lucky. I had an excellent local library growing up. I had the kind of parents who could be persuaded to drop me off in the library on their way to work in summer holidays, and the kind of librarians who did not mind a small, unaccompanied boy heading back into the children's library every morning and working his way through the card catalogue, looking for books with ghosts or magic or rockets in them, looking for vampires or detectives or witches or wonders. And when I had finished reading the children's' library I began on the adult books. We have an obligation to support libraries. To use libraries, to encourage others to use libraries, to protest the closure of libraries. If you do not value libraries then you do not value information or culture or wisdom. You are silencing the voices of the past and you are damaging the future. We all – adults and children, writers and readers – have an obligation to daydream. We have an obligation to imagine. It is easy to pretend that nobody can change anything, that we are in a world in which society is huge and the individual is less than nothing: an atom in a wall, a grain of rice in a rice field. But the truth is, individuals change their world over and over, individuals make the future, and they do it by imagining that things can be different.

• This is an edited version of Neil Gaiman's lecture for the Reading Agency.