“The question I am most often asked is how did I get started in this business. And when I’m asked that question I always tell this story, and you can believe it because it is a true story, but then I am a pathological liar. But not really.”

So began the patter that Harry Anderson often used to introduce his signature trick, his show closer, the Monarch Monte. It was a simple trick, really. Three gigantic playing cards were shuffled and swapped places, similar to the street hustler game Three-Card Monte. What elevated the trick to a place of prominence was the story that Anderson told while he performed it—a story of a con man teaching a young Anderson not only the “religious experience” of swindling people of their money and leaving them happy, but some of the most important lessons of his life. That respect is something you give long before you get, that trust is something you save for yourself, that you never eat at a place called Mom’s or shoot dice with a guy called Pops, and that a fool and his money were lucky to get together in the first place.

Most people know Harry Anderson as Judge Harold T. Stone, the affable star of NBC’s Night Court, which ran for nine seasons from 1984 to 1992. To Anderson, however, playing Judge Stone was just a job. He never set out to be an actor, and his arrival at the center of a hit sitcom was something of an accident. Harry Anderson was no actor. He was a magician. He was a comedian. He was a storyteller and a showman. At his heart, however, Harry Anderson was a hustler.

Anderson was born in 1952 in Newport, Rhode Island, into an unhappy home. His father, a salesman, left him and his mother alone. Anderson moved around a lot in his younger years, living in a dozen states by the time he was 16 years old. He and his mother moved to Las Vegas, where she took a job as a casino dealer, then to Chicago, where she worked as a prostitute —a fact about which Anderson has been blunt and refused to hold against her. “She did what needed to be done to try to keep us together,” he told People magazine. In Chicago, Anderson spent many days alone in the lobby of the Ambassador East Hotel, a popular spot for men to play cards and gamble. The card players would look after Anderson while his mother was busy, teaching him to play games, make wagers, and perform card tricks.

By the time he was 16, Anderson left Chicago to live with his father in Los Angeles. He took up magic as a serious hobby during high school, and joined a club in which he socialized with other future magic stars, like Jonathan Pendragon and Paul Harris. During the summer of his senior year at North Hollywood High School, Anderson visited San Francisco. It was here that he first saw a street hustler performing the shell game. For Anderson it was a revelation —he could put his magic skills to work and earn some real money! He developed his own shell game routine and hit the streets, though according to Anderson he was impeded by not having a crew. He had to hustle “against the wall,” meaning he had no shills to interrupt the game, to place winning bets when the crowd grew tired of losing, or to look out for the police or trouble. After Anderson had his jaw broken by an upset customer, he reworked his shell game into an act instead of a con. He performed it as a parody of a street hustler and kept a hat out for donations rather than taking bets on where the pea was each time. He channeled the men he played cards with in the Ambassador East Hotel in his youth and adopted a persona he called “Harry the Hat,” a fast-talking con man who was always one step ahead of the audience but still “a bit of a nincompoop.” “I’ve played cards with people who don’t share a language, but we can play cards,” he told The Globe and Mail. “It was out of admiration for them that I decided to become a magician. I think that’s where Harry the Hat came from.”

After graduating as valedictorian in 1970, Anderson moved to Ashland, Oregon, and opened up a magic shop. During the summers he performed magic at the Oregon Shakespeare Festival. Each year Anderson would develop a new effect to perform, and he would build his show around that effect. One year he upstaged the festival’s performance of Henry V with his own version of the Bullet Catch where he had his assistant shoot a gun at him and he caught the bullet with his teeth. When the festival wrapped, Anderson hit the road to perform on street corners all over the country, from San Francisco to Texas to New Orleans to New York City. He would often show up in college towns and perform his entire act in the middle of campus for free, merely passing the hat at the end.

Word got around the magic community about Harry’s act, and he was invited to perform at the vaunted Magic Castle in Los Angeles. Throughout the early 1980s, Anderson would come to L.A. for one week each year to perform in the Castle before disappearing again and hitting the road to work the streets. He eschewed nightclubs and theaters for street corners and parks. His act involved things like cutting off his arm and snapping his fingers in animal traps. His character told hilarious stories of swindlers and hucksters, carnival barkers and sideshow freaks.

One of Anderson’s signature pieces was a trick called the Needle Through the Arm. It was a simple feat; he stuck a long hat pin through his arm. But like most of Anderson’s act, he drew out the trick into a 10-minute soliloquy about the history of circus geeks and how the sublime and profane was entertaining. “It’s not like when you put the girl in the box and you cut the box in half,” he would explain to the audience. “Because when you see that, you know that it couldn’t be happening. But when you see the Needle Through the Arm, you know that it could be happening. It ain’t. But it could.” After one performance at the Magic Castle in 1977, an executive from CBS who had come to see the show left in disgust. “Who the hell is that guy and why do you let him do that?” the executive said to Milt Larsen, the club’s founder. “I would never hire that guy.”

In 1980, Anderson finally got his break when a talent manager saw his show at the Magic Castle and asked him to open up for Kenny Rogers in Las Vegas for $1,500 a week. From there, Anderson was booked on The Mike Douglas Show, The Merv Griffin Show, and Saturday Night Live. On his third guest spot on SNL, he performed the Needle Through the Arm on live television for millions of people. The audience roared in revulsion and disgust, even though he constantly repeated throughout the act that it was just a trick, “a gag.” It didn’t matter. This was the genius of Harry Anderson’s act, the thing he took from the con men and street hustlers of his youth. If you’re going to take their money, make sure they’re happy, lest you wind up with a broken jaw. Most magicians told the audience they were doing something magical, and the audience didn’t want to believe it. Harry the Hat told the audience he was lying to them, but the audience wanted to believe it was real.

Les Charles, the producer of Taxi, had a deal with NBC to make a show called Cheers. He had seen Anderson perform on the street as Harry the Hat and thought that a con man character could work on the show. They signed Anderson to perform on three episodes during the first season. His character would show up in the bar and run small cons on the patrons, only to be chased off by the bartender, Sam. It was a small part, but enough to get him noticed by Reinhold Weege, who was producing a show called Night Court. Weege asked Anderson if he’d read for the main character—a young judge who did magic tricks and romanticized the music and style of his parent’s generation. Anderson told Weege, “I am this guy. I’m exactly who you’ve written here.”

Night Court was part of what is considered a golden age for television sitcoms, and especially so for NBC. The show was part of a Thursday-night lineup that included The Cosby Show and Cheers. Night Court was a hit, and Harry Anderson became a celebrity. It stuck with Anderson, however, that he wasn’t famous for what he knew he was best at. “I was never really an actor,” he told WGN in 2014. “I was a magician who fell into a part on ‘Cheers.’” Not wanting the public to forget about his roots, nearly every time Anderson was asked to appear on a talk show, he performed magic. When he was asked to host Saturday Night Live in 1985, he didn’t deliver a monologue. He performed a trick where he ate a live guinea pig. In the middle of the show he performed an esoteric and slow chapeaugraphy routine—a French panhandler act using a single piece of felt that dates back to the 1600s. During one sketch in the show, Anderson plays himself shooting magician Doug Henning in the head and killing him. They were odd choices for the show, but Anderson had earned enough clout with NBC and enough respect from Dick Ebersol, then SNL’s producer, that he was allowed to call his own shot.

After Night Court was canceled in 1992, Anderson was given the lead role in a CBS sitcom called Dave’s World based on the life of syndicated newspaper columnist Dave Barry. That show lasted four years, and when it was canceled, Anderson no longer wanted to chase acting roles. He was never a fan of Los Angeles. He moved his wife and children to a small town in Oregon and would fly back and forth to film Dave’s World. Being famous just didn’t suit him. “To me, it’s a little bit embarrassing to be in the public eye that much. People start asking me about what I think about the situation in the Middle East … just because I’m in the public eye … as if I have some kind of knowledge or opinion that matters,” he told LH Magazine. “It’s embarrassing. And after all those years I spent on the street, anonymous, and with intended anonymity, that was a jolt that I never quite adjusted to.”

By 2000 he had stopped acting in TV and movies almost entirely. He longed to return to performing magic. He looked at his former magic peers with envy. While Anderson was acting in sitcoms, Penn & Teller, who came up around the same time as Anderson with a similar act and sensibility, had become two of the biggest names in magic and had reinvented what the genre could be. “When Harry sees what Penn & Teller have attained, he is envious,” Anderson’s friend Mike Caveney writes in his memoir of Anderson. “Harry is faced with a generation of people who are surprised to find out he is a magician.”

“I am richer than Davy Crockett,” Anderson told People in 2002. “I can settle back and do what I want to do. And what I want to do is card tricks and magic.” Anderson purchased a number of properties in New Orleans’s French Quarter, near where he once lived in the 1970s while performing in Jackson Square. He opened a magic shop called Sideshow. He bought a building where Lee Harvey Oswald once lived and opened up a nightclub called Oswald’s Speakeasy. It was in this club that Anderson started putting his magic act back together, performing his 90-minute one-man show five nights a week for tourists and colorful locals alike. He also opened Spade & Archer, named for the detective agency from The Maltese Falcon, which served as his appointment-only showroom for his unrivaled collection of carnival, magic, and gambling memorabilia (including the original Maltese Falcon, a prop Harry loved to say he lifted from the Warner Bros. lot).

Anderson quickly became a favorite among the community of street performers and artists that called the French Quarter their home. The reclusive Anderson had found people in New Orleans that he could relate to and often opened up his home to those he dubbed the “Quarter Rats.” “What’s been constant here is the acceptance, the way this place accepts you however you come in,” Anderson told The Globe and Mail in 2003.

When Katrina hit in 2005, the French Quarter was devastated. Businesses like Anderson’s depended on tourists, and they were in short supply after the hurricane. More importantly, for those who made the French Quarter their home, there were virtually no public services in the wake of Katrina. No one was picking up the garbage. In the weeks right after the storm, there was no electricity, no running water. It was bedlam. Anderson decided to do something about it. He opened up Oswald’s as a gathering spot for people in the community to plan their recovery. The weekly town hall meetings in Oswald’s began as small gatherings around the bar by candlelight, the power still not restored. Eventually they grew to large and raucous events that drew important civic leaders. Officials from FEMA, the Army Corps of Engineers, even the City Council knew that if it wanted to speak to residents in the French Quarter, it had to face Harry Anderson and the Quarter Rats at Oswald’s. Where the city’s leadership failed, the locals at Oswald’s would take up the charge themselves, organizing trash pickups and brainstorming solutions to the myriad problems that had beset them after the storm.

In 2006, Anderson was attacked in the French Quarter, the second time he had been assaulted since living in New Orleans. Soon after, Ray Nagin was reelected mayor. “That was the nail in the coffin,” Anderson told The New York Times. He and his wife decided to leave the city they loved. They sold their properties and moved to Asheville, North Carolina, where they lived quiet, reclusive lives until Anderson died Monday, reportedly of natural causes. He was 65 years old. He is survived by two children, Dashiell and Eva Fay, and his wife, Elizabeth Morgan.

Harry Anderson leaves behind a legacy that is as diverse as the many hats he wore, both literally and figuratively, throughout his life. To fans of his television show, he was warm, funny, and endearing. To New Orleanians, he was a civic leader. To magicians, he was an innovator and a trailblazer. Through it all, he was Harry the Hat, a character of his own invention, a myth of his own making. How much of his life story was real and how much of it was a tall tale, we may never know. For a man who claimed to never be an actor, Harry Anderson did plenty of acting. He acted as someone else. He acted with confidence even when afraid. He managed to cultivate an iconic character that eventually fused with his actual persona.

At the conclusion of the Monarch Monte routine in Anderson’s old stage show, he would say, “I gave this game a lot of thought over the years since I saw it the first time. I wondered why in the hell do the marks keep running to play a game they haven’t beaten in over 5,000 years? All I could conclude was it was so damn simple it was irresistible. … And it isn’t really the cards you’re up against. It’s the man who shuffles those pasteboards. He’s the one you’ve got to keep your eye on.” The real Harry Anderson was private, someone most of us never knew. Harry Anderson wasn’t Harry the Hat. He told us as much. But like a great hustler, he knew we wouldn’t want to believe it.