Hefner lived in the iconic Playboy Mansion until the time of his death

Hugh Hefner, the founder of Playboy magazine and star of E! reality show The Girls Next Door, has died, PEOPLE confirms. He was 91.

Born Hugh Marston Hefner on April 9, 1926 in Chicago, Illinois to parents Grace Caroline and Glenn Lucius Hefner, the businessman died on Wednesday.

Get push notifications with news, features and more.

“Hugh M. Hefner, the American icon who in 1953 introduced the world to Playboy magazine and built the company into one of the most recognizable American global brands in history, peacefully passed away today from natural causes at his home, The Playboy Mansion, surrounded by loved ones,” a rep for the Playboy Enterprises founder said in a statement to PEOPLE.

“My father lived an exceptional and impactful life as a media and cultural pioneer and a leading voice behind some of the most significant social and cultural movements of our time in advocating free speech, civil rights and sexual freedom. He defined a lifestyle and ethos that lie at the heart of the Playboy brand, one of the most recognizable and enduring in history. He will be greatly missed by many, including his wife Crystal, my sister Christie and my brothers David and Marston, and all of us at Playboy Enterprises,” said Cooper Hefner, his son and Chief Creative Officer of Playboy Enterprises.

RELATED VIDEO: Fans Head To Playboy Mansion To Pay Respects To Late Hugh Hefner: “He Made It Ok To Be Yourself”

After serving in the army and attending college, Hefner got the idea for Playboy while working in the publishing industry.

Image zoom AP Photo/Kristian Dowling

By putting up his furniture as collateral for a loan and borrowing the rest from family and friends, Mr. Hefner published the very first issue of Playboy in December of 1953, which featured a nude Marilyn Monroe.

The same year, Hefner launched media and lifestyle company Playboy Enterprises, Inc., on which he served as a board member until the time of his death.

Of Playboy’s iconic logo, Hefner said in a 1967 interview: “The rabbit, the bunny, in America has a sexual meaning; and I chose it because it’s a fresh animal, shy, vivacious, jumping — sexy.”

Image zoom Playboy

Image zoom Playboy

The magazine became known for its articles as well as the beautiful women that graced its pages, with Hefner asking some of the world’s greatest and most progress literary figures to write for him including, Hunter S. Thompson, John Updike, Ian Fleming, Joseph Heller, Gabriel Garcia Marquez, Margaret Atwood, Jack Kerouac and Kurt Vonnegut.

Following his decade-long marriage to first wife Mildred Williams, with whom he shared daughter Christie and son David and whom he divorced in 1959, Hefner was public about sharing intimacies with numerous women.

In 1971, Hefner purchased the legendary Playboy Mansion in Los Angeles where he relocated to permanently and lived until the time of his passing.

After suffering a mild stroke in 1985, his daughter Christie assumed more responsibility in the management of the company (she was named president in 1982 and CEO in 1988 until she left in 2009). In 1989, he tied the knot with second wife and Playmate of the Year Kimberley Conrad. The couple had two children together, sons Marston Glenn and Cooper Bradford, before separating almost a decade later and officially divorcing in 2010.

Throughout his the five decades of life in the Playboy Mansion, Hefner was recognized for housing multiple young women in the home at the same time, including Girls Next Door stars Holly Madison, Bridget Marquardt and Kendra Wilkinson. The six-season E! reality series documented the lives of the Playmates and their relationship with Hefner inside the mansion.

Image zoom Denise Truscello/WireImage

After his seven-year relationship with “No. 1 girlfriend” Madison ended, Hefner dated a succession of women before falling in love with Crystal Harris, whom he married on Dec. 31, 2012. In April, Hefner’s younger brother Keith died at the age of 87 from cancer.

Known for hosting its annual Playboy Halloween party, the Playboy Mansion sold to Hefner’s neighbor Daren Metropoulos for $200 million in June with the stipulation that Hefner was able to live in the home until his death.

Image zoom Playboy

In an August 2017 interview, Hefner’s son Cooper, who took over as chief creative officer of Playboy Enterprises, admitted that it was hard to watch his father get older.

“It’s tough to watch him struggle,” he told The Hollywood Reporter of his father getting older. “But I’m just happy it’s physical and not mental.