George R.R. Martin spent a decade in television before realizing he wanted nothing more to do with it. The fantasy author, who had won multiple Hugo and Nebula Awards for his fiction in the 1970s and early 1980s, had grown tired of having budgetary restrictions limit the scope of his work. And if there was money, there was little time: As a staff writer and producer for the CBS series Beauty and the Beast, Martin knew he had 46 minutes to tell a cogent story. It wasn’t nearly enough.

Calling it a “reaction” to his 10 years in TV, in 1991 Martin began to sketch out a series of fantasy novels that would feature hundreds of characters, sprawling scope, and no network ceiling. When the 704-page A Game of Thrones hit shelves on August 1, 1996, it received positive notices and respectable sales, but there was little hint of the hysteria that would follow.

Born in 1948, Martin attended Northwestern University and pursued degrees in journalism. At 21, he sold his first piece of fiction to the anthology magazine Galaxy; a novel, Dying of the Light, followed in 1977. His fantasy was wide in scope, but had elements of the mundane to ground his stories. Other writers, Martin once observed, embraced the bigger tropes but didn’t "have the dogs in the halls of the castles scrapping under the table.” Martin preferred a lived-in approach.

In 1985, Martin was engaged as a staff writer for the CBS revival of The Twilight Zone and got his first taste of the compromises of television. Adapting a Roger Zelazny story titled “The Last Defender of Camelot,” Martin had scripted a faithful recreation of the climax where a swordsman is fighting against an enchanted, empty suit of armor near Stonehenge.

“Then the line producer calls me in and says, you can have Stonehenge or you can have horses, but you cannot have horses and Stonehenge,” Martin told TIME in 2011, “because there is no Stonehenge around here and we’re going to have to build that out of papier-mâché on the sound stage. And if we bring horses there, when they start galloping around, the rocks will shake and fall down.”

The practical demands of TV eventually wore on Martin’s nerves—even worse was the fact that he could toil on a project and then not even see it produced. On the development deals following his work on Beauty and the Beast, Martin told January magazine that “nothing ever got made … It was one of the things that ultimately frustrated me and drove me back to books.”

In 1991, Martin was beginning work on an unrelated science-fiction title, Avalon, when an idea popped into his head: dire wolf puppies in a summer snow. An entire chapter followed, and Martin was soon working on the book every spare moment he could. In 1993, he wrote a letter to his agent updating him on what he was now calling A Game of Thrones:

“The enmity between the great houses of Lannister and Stark [will play] out in a cycle of plot, counterplot, ambition, murder, and revenge, with the iron throne of the Seven Kingdoms as the ultimate prize … a second and greater threat … where the Dothraki horselords mass their barbarian hordes for a great invasion of the Seven Kingdoms, led by the fierce and beautiful Daenerys Stormborn … where half-forgotten demons out of legend, the inhuman others, raise cold legions of the undead and the neverborn.”

At the time, Martin thought a trilogy would be sufficient to tell his story, which is how the project was shopped. Of the four publishers interested, Bantam Books made the best offer, assigning editor Anne Groell to the series. Groell, who had seen some of Martin’s work while it was in circulation, found that even non-fantasy fans working at the publishing house were talking about it. A Game of Thrones seemed like a sure thing. And if it wasn't, Martin figured it wouldn't take much time to complete. "I ought to finish ... by 1998," he told an Omni Visions chat room in 1996.

Getty Images

While sales were respectable for Bantam’s first printing—known by collectors for its silver-foil cover—A Game of Thrones was not a commercial hit. When Martin showed up for book signings, some stores would be virtually empty. At one Dallas location, Martin perked up when he saw hundreds of people lining up. But he soon realized they were after a new title in the Clifford, the Big Red Dog series.

Readers and independent booksellers continued to pass around copies to friends, and those friends handed them over to others in a kind of literary proselytizing. As Martin added to the saga—three books became five, then a plan for seven—the audience grew. By 2011, more than 15 million copies of A Song of Ice and Fire, the blanket name for the series, had been sold.

In 2000, Martin was asked if the reason he returned to books—the spotty predictability of film and television—might one day lead him back by offering to adapt Thrones. “I have had some interest in the book, yes,” he answered. “I don't know if anything is going to come of it.”