You’ve heard it before, and no doubt you’ll hear it again: Cosby sweaters just won’t go away. The ubiquitous gaudy knits, named after those worn by comedian Bill Cosby on “The Cosby Show,” are coveted by everyone from irony-laden thrifters to fashion’s top designers. Not unlike the ugly Christmas sweater trend, Cosby’s flamboyant knitwear remains a cultural touchstone. But beyond our nostalgia for the ostentatious ’80s, the reasons why the world went Cosby-sweater crazy—even as their namesake’s reputation crumbled—are less clear. So back in 2013, shortly before Cosby’s very public fall from grace (during which his choice of sweater didn’t go unnoticed by one keen-eyed observer), we asked Dr. Cosby about his signature garment.

“We’re talking about the knit woolen things that look like the sheep were different colors or fell in some paint, right?” Cosby says over the phone. “Isn’t that what you’re talking about?”

Exactly. Besides referring to the sweaters sported by Cosby’s character, Dr. Heathcliff Huxtable, to invoke the phrase “Cosby sweater” is to call something garish, tacky, and outdated—in an affectionate way. And in the cyclical world of fashion design, such passé looks have recently inspired a swath of contemporary looks featuring bold, geometric patterns and incorporating a mishmash of colors and textures.

Cosby himself doesn’t even remember when he first heard the term “Cosby sweater,” and is hard-pressed to explain why the style is such a phenomenon among young people today.

“I have no idea, and I’m not going to make up anything,” says Cosby, “but I think youthful people have a long time to live, so they can waste some time on something like that.” If you do have some time to waste, check out popular blogs like Huxtable Hotness and the Cosby Sweater Project, which document the intricacies of “The Cosby Show” style, breaking down the fascination one sweater at a time.

Though writing the sweater phenomenon off as another meme-driven, nostalgic obsession doesn’t really hold up when you start watching reruns. The more episodes you see, the more you realize that Cosby actually wore an insanely large collection of kaleidoscopic sweaters. What’s the deal?

“He said, like Betsy Ross to George Washington, ‘I will make you a sweater.'”

In fact, Cosby adopted the fuzzy fashions out of necessity: Costume designer Sarah Lemire, who worked with Cosby from his sweater-vest Jell-O ad days, says that initially, she had various suits made for Dr. Huxtable to wear. They quickly realized that Cosby, and by extension Dr. Huxtable, couldn’t really be at ease wearing a suit around the house. “Bill basically likes to be comfortable, and in his real life, he’s in his sweats or his PJs,” says Lemire.

Avoiding straitlaced, white-collar attire also made Dr. Huxtable a more dynamic character. “I wanted to get away from the white coat all the time,” says Cosby, “or the blue blazer look, with the khaki pants and the penny loafers.” In contrast, Dr. Huxtable’s sweaters infused the show with a contagious, creative energy.

This atmosphere was also fostered by the unique production environment for “The Cosby Show.” Director Jay Sandrich wanted to capitalize on Cosby’s ability to improvise and build off his audience’s reactions by filming the program in two separate takes with a live studio audience, then selecting the best shots from each take. Lemire says that Cosby often strayed from the script, following his gut if he thought it might get a better laugh. “It was incredible and it came out of nowhere, and the director knew to grab that.”

As a result, the show often relied on close-up shots of Cosby to capture such moments of improvised humor. However, tight shots like these caused problems when matching the scenes from two different takes, as a slight difference in costume positioning would become a glaring mistake.

“Usually you don’t do close-ups on TV, and that’s why we started using sweaters,” says Lemire. “As our bodies move around, the clothes are going to shift between the first and second take. If you have a jacket on, and the shirt collar’s in one spot, it’s going to slide off a little on one side or the other, or it might do something else that didn’t match. Sandrich was a real stickler for things matching, so we just did the sweater thing. I actually sewed his shirts to the sweaters so that nothing moved.”

Once they settled upon sweaters as the costume of choice, selecting a print was equally complicated: The available camera technology meant that certain patterns and textures had to be carefully avoided. “The show was shot with multi-cameras,” says Lemire, “and back then they had a lot of problems with strobing, so it was very difficult to use certain patterns.” The stockinette stitch, a standard on most sweaters, alternates rows of knitted and purled stitches, which results in a subtle ribbing or stripe effect. The cameras used for “The Cosby Show” made even solid-colored stockinette sweaters vibrate or strobe when onscreen.

Director Sandrich was also adamant about giving each character a different look. “We had seven people on camera at the same time,” says Lemire, “and he didn’t want two people to wear the same color. Well, that’s a little difficult to have seven different colors out there, so we started doing patterns. ”

Thus Lemire turned to the complicated prints that made up much of the fashion lexicon during the 1980s. “The predominant company that made Bill’s sweaters was Perry Ellis, and that’s because they didn’t cost too much money, they used a flatter knit, and they would have patterns, meaning I could have color there without Bill being in a solid color, which would segue into whatever Phylicia Rashād or the kids were wearing.”

Though today, any kooky sweater might be called a “Cosby sweater,” there was one brand whose garish style didn’t quite make the show’s cut—the Australian label Coogi. “My sweaters were busy to a certain point, but it wasn’t to that extreme,” says Lemire. “I still can’t stand those.”

So where did Lemire find all these fabulous garments? “I had very little money to use, and it never grew to very much, so I borrowed stuff. The garment district, and Missoni in particular, was wonderful about loaning them.” Ultimately, Cosby’s sweaters came from all kinds of sources, ranging from mainstream department store labels to handmade, one-of-a-kind items.

Early in the show’s run, an architecture student in Boston sent Lemire photographs of some interesting pieces she had knitted, which began a collaborative relationship. “Because Bill runs track, and we did an episode where we shot him running at the Penn Relays, she made one with Bill running on the sweater,” explains Lemire. “Another she did was the skyline of New York City, with fireworks above it. Those were both sweaters that I designed and she knitted.”

Perhaps the most iconic Cosby sweaters were those designed by Koos Van Den Akker, which were actually discovered by Cosby, rather than his costume designer. Though Mr. Huxtable wore many sweaters with wild colors and patterns, Van Den Akker’s designs raised the bar by incorporating a pastiche of different textures as well.

“And in those days you also got shoulder pads, for free.”

Raised in Holland, Van Den Akker came to New York via Paris in 1968, after having worked as an apprentice at Christian Dior. In 1970, he opened his first store on Columbus Avenue with his signature collaged women’s wear, and the curious reviews followed soon after. “They had no idea what my stuff was about,” says Van Den Akker, “whether it was home sewing or something more than that. Most of the time they went with home sewing, but it didn’t matter. People had never seen this kind of stuff, so it was weird and strange and different.”

By the late 1970s, Van Den Akker’s designs for his eponymous KOOS label were sold in top department stores and sported by trend-setting celebrities. One of those stars was the singer and actress Josephine Premice, whose unconventional garments apparently caught the notice of Mr. Cosby and his wife, Camille, in the mid ’80s.

“Josephine was very tastefully flamboyant, and Mrs. Cosby, my un-flamboyant wife, noticed and loved some of the things she was wearing,” says Cosby. “I remember Josephine saying to Camille, ‘I had a fur coat, and it was so old,’ Josephine laughed, ‘that there were holes worn in it, and there was nothing I could do.’ Koos said to her, ‘Bring it to me and I’ll make a coat for you with the pieces.’ Whereupon, with his genius mind he began to mathematically put together the best pieces of the coat along with other fabrics. He styled the coat into a new invention that was eye-catching and tasteful and had women going after Josephine, to hit her in the head and take the coat off of her. Because you know that’s the way women are; they will do that.”

Premice took Mrs. Cosby to get her own coat repaired by Van Den Akker, and the result thoroughly impressed Mr. Cosby. “Geometric is the word,” he says, “the genius of geometric working pieces and cuts.” Shortly thereafter, Cosby received his first KOOS sweater as a birthday gift from Premice, around the time when “The Cosby Show” debuted in the fall of 1984.

Van Den Akker admits that this first garment was just an ordinary, extra-large women’s sweater he had hanging in his show room. “She just took it off the rack, this big women’s size, and took it to Bill. He put it on, and it looked great, and he had to go on camera right away so he kept it on, and that is how it began.”

Eventually, Cosby went to Van Den Akker’s shop himself to seal the deal. “You know Betsy Ross?” asks Cosby. “She said to George Washington, ‘Let me make you a flag or sew you a flag,’ or something. Well this is how Koos’ sweaters started with me. He said, like Betsy Ross, ‘I will make you a sweater.'” Besides the eye-catching patterns, Cosby was delighted with Van Den Akker’s inventive collages of high-quality fabrics, including wool, silk, leather, and camel hair. “And in those days you also got shoulder pads, for free,” Cosby adds.

“I used all kinds of materials,” says Van Den Akker, “but it’s more about the colors. I like muddy colors and moody stuff. I love it all. My workroom is one big pile of fabrics; it’s a huge mess and it’s wonderful. Today everybody is doing collaged clothing, but in the ’70s and ’80s, I was alone. I was the only one.”

However, Lemire points out that Van Den Akker’s typical collages didn’t work so well under “The Cosby Show” filming conditions. “Van Den Akker’s work is fantastic, but he used patterns that would not have worked on camera, so I’d go over and talk with him and we’d put together the ones that would work, and he would make a sweater where it was something I could use.” As a result, Van Den Akker’s sweaters appeared more during the introductory credits than in actual episodes. “I could do them in the opening titles, because that was done on film, and later transferred to tape,” says Lemire. “So you see them in the opening titles more than anyplace else.”

Regardless, Van Den Akker loves that his reputation is tied to the gaudy sweaters worn by Dr. Huxtable. “You know that movie ‘High Fidelity’ with John Cusack?” Van Den Akker asks. “Well, I think that was the first time I heard the term ‘Cosby sweater,’ and it was so much fun. There have been jokes about the sweaters in the ‘New Yorker’ and places like that, and it’s always negative. A Bill Cosby sweater stands for ‘crazy,’ and I love it.”

If anything, the mutual admiration between Van Den Akker and Cosby has grown, rather than diminished, over the years. “To me, those sweaters meant, obviously, one-of-a-kind collector’s items,” says Cosby. “They were like works of art, like a walking piece of art.”

Today, as an aging generation of customers empties their closets, Van Den Akker tells them to bring these older designs back to him, and they can be re-sold on the vintage rack in his store on Madison Ave. As for Mr. Cosby, does he still have any of his classic sweaters stashed away somewhere? “I have no idea what I have,” says Cosby. “I’m married 49 years, and all I know is I have one drawer left, and I don’t where the rest of my stuff is. I have a feeling, and some people say it sounds cruel, but I have a feeling upon my death, some 20 minutes after, eBay will explode.”

(To go behind the scenes at Koos Van Den Akker’s studio, check out his profile by VICE magazine. This story was updated in January of 2016.)