Stephen Colbert has spent the last few weeks saying goodbye to some of his best recurring segments and running gags: Better Know a District, The Word, Formidable Opponent. But it’s oddly fitting that the last non-fictional guest on “The Colbert Report” (Thursday night’s guest is Grimmy the Grim Reaper, who may or may not be there to formally kill off the Colbert persona) will be an author. And not just any author: the National Book Award-winning author of a book of short stories about war—basically the least TV-friendly kind of writer possible.

When Colbert interviews Phil Klay on Wednesday night’s penultimate episode, he’ll be concluding his role as one of pop culture’s greatest literary boosters. It was a position Colbert adopted explicitly during the Hachette v. Amazon fight this summer, when he became one of the publishing industry’s most visible defenders. He attacked Amazon on air, then championed debut Hachette novelist Edan Lepucki, turning her book California into an unlikely bestseller. (Colbert had a dog in the fight: His own book is published by Hachette.)

Since premiering nine years ago, “The Colbert Report” joined “The Daily Show” as one of the best places to hear historians, economists, and journalists discussing their work—and the best place for non-fiction authors to plug their new books. In the last few years, though, the show also began to host more fiction writers. It’s unclear whether this will continue in Colbert’s new home; clearly the real Colbert loves books, but a network late-night show host has certain other responsibilities, like interviewing celebrities on the Hollywood promotional tour. In the meantime, here are some of Colbert’s most delightful and thoughtful literary interviews.

In 2012, Colbert prepared for the release of his own children’s book with a two-part interview with Maurice Sendak. At 83, Sendak was crotchety and endearing, huffing markers with Stephen and telling us what he really thought about e-books: “They cannot be the future. They may well be. I will be dead. I won’t give a shit.” Sendak died a few months later.

In this 2013 interview with George Saunders, Colbert proves that asking silly questions—“Why do you write short stories? America likes big. Go big or go home.”—can yield smart, insightful answers.