LightBox Newsletter Sign up to receive the latest from TIME's photo editors. View Sample SIGN UP NOW

At first it seems a supreme irony: Larry Fink, the photographer best known for documenting America’s upper classes, makes no secret of his communist background. But that side of his life is not the whole story, and his five-decade-long career makes that very clear. “My mother was a party member for a while,” he told TIME ahead of the LOOK3 festival in Charlottesville, Va., where his work will be on show starting June 10. “She was also a minxist — she liked to wear minks. She loved fancy things and my father and she used to go off jazzing and partying.”

This dual background — “I was inculcated with an ideological disdain towards the upper class, and yet, at the same time, a luring involvement with the necessity of pleasure and luxury,” he said — has informed and influenced much of his work.

His crude yet carefully composed black-and-white photographs, of rich men and women celebrating their wealth and power at lavish parties, form an essential document of a slice of American society that is often ignored by photojournalists who are inclined to document the powerless.

Fink’s choice to photograph the rich versus the poor came with drawbacks. For a long time, he was perceived as a party photographer — a “nobody at all,” as he put it. But his work wasn't the typical party photography, enabling him to carve out a more important niche: “Since I photographed a little bit more obsessively and not quite so flatteringly, I had to make up a legacy for myself. I would tell [the rich] that I’m photographing the archives of gaiety. That was my lesson plan for liberty: the archives of gaiety. So they perceived me as a friendly guy, and I was.”

Today, history has given Fink's work a new layer of meaning. “I’ve been noticed as a major player in the world of photography, and I like that,” he said. “But nothing is ever complete. This archive is a massive swelter of energy. My stuff is a sort of passing-through document. It’s an archive of my own immersion in gaiety."

Fink’s photographs produce a mix of emotions amid his audience — often including disgust or anger toward the ostentatious display of wealth. Those are sentiments he doesn’t share. “I don’t have any rabid Marxist or leftist hostility towards rich people,” he said. “I have what might be called ordinary envy, like any of us who wouldn’t mind [having] a private plane or a big yacht. But it doesn’t make me have disdain for anybody that has these privileges.”

Fink likes all kinds of people — except the sociopathic kind, he’s quick to point out — so his pictures “look, if not kind, at least even, receptive, emphatic,” he said. “The empathy factor is interesting. If you live broadly and are curious about it all, you understand what it is about us that draws us together. It’s that kind of picture that I really like to make.”

That empathy, if not sympathy, comes from the fact that, as human beings, “even the most powerful of us are being dominated by very small, primary and unresolved human impulses of jealousy, greed, insecurity and so on,” Fink said. “We’re all human, even the guy with the jet. We’re all infinitely vulnerable. With that thing in mind, we have to look at people with some degree of patience and kindness, and also with curiosity. Because every time you meet somebody, more than likely, you’ll learn something.”

This is what Fink has spent the last 40 years telling his students in his various teaching jobs at the New School, Yale, New York University and, since 1986, at Bard College in Annandale-on-Hudson, N.Y. “Photography is a marvelous way to empower people’s curiosity,” he said. “It’s the critical instrument of the curious.”

Untitled, 1958. © Larry Fink Untitled, 1958.

© Larry Fink 1 of 12

He laments the loss of curiosity among new generations of photographers. “It’s sad for me to see a lot of young photographers [adopt] these post-modernism strategies with all the projects that are based on self-perpetuated fantasies. We’re living in an age where it’s all about career; it’s all about narcissism,” he said, mentioning the growing number of photographers looking for gallery representation. “In the old days, when money was not quite so polluting as it is today, the idea of courting curators was repulsive. The idea, at the time, was that if you did get yourself into a gallery, it somehow meant that your work had lost its edge. Now the younger artists, and I would not want to malign their sincerity, nor would I want to take away their fun or their heroic understanding of what it means to be alive in this time, are just opportunists for the most part. It’s about making art as a career.”

Yet Fink remains hopeful, believing that curiosity can be stimulated among these younger photographers. “You just have to crush their toes with stones, punch them in the nose with bones, open their eyes," he said. "[You] just try to open up the inside to see whether or not they will be receptive to the outside.”

Larry Fink 's photographs will be on show in Charlottesville, Va. until June 30. The photographer will give a talk on June 11 during the LOOK3 photo festival, which runs from June 10 to 13.

Olivier Laurent is the editor of TIME LightBox. Follow him on Twitter and Instagram @olivierclaurent