“It’s incredible, right?” shouts Jeff Greene over the roar of the two-seater dune buggy’s motor. “It’s 55 acres!” Still in his whites from this morning’s tennis match, he’s giving a personal tour of his Sag Harbor estate, barreling at 30 miles per hour through the vast forest of scrubby pines and soft moss of its gated grounds. “Beautiful nature here!” A blur of deer goes by, and the trees break to reveal the summer sun glinting off a grassy lagoon. Greene slows by its shore. “This is our swan pond, and this is our private beach,” he says, gesturing toward a slip of white sand encircling the edge of the North Haven Peninsula. “It goes all the way to the ferry. Three thousand feet of beach,” he adds, a smile spreading across his tanned face.

Greene made his fortune in real estate, and he’s never been shy about showing it off. “Having money is great,” he says. “It’s fun. The more the better.” For years, he was a fixture on the party circuit, and celebrities like Lindsay Lohan and his old roommate Heidi Fleiss were not uncommon sights on his yacht, The Summerwind. In 2006, after an old friend, John Paulson, showed him how he was planning to short the impending housing bust, he replicated the trade and made himself a billionaire. Since then he’s settled down, married a bubbly Chinese-Australian woman named Mei-Sze, had two children, and changed positions once again. “I’ve got a huge, huge position in mortgage-backed securities,” he says. “I started accumulating them in 2009, when the market was really down and things were really scary.” That’s also when he picked up this property for around $40 million (half the 2007 listing price), which he and his wife have christened “Greene Haven.” “I wish we could spend more time here,” he says. “Honestly, we have so many great homes.”

He cuts the engine, and for a moment the only sound is the waves lapping peacefully against the shore. Greene gazes across the bay at the multi-million-dollar houses peeking from behind the trees. I assume he’s quietly contemplating acquiring even more of the shoreline, but then he says something surprising. “If somebody wanted to go after a rich person,” he observes, “they have got their pick of the litter out here.”

It’s strange to imagine someone like Greene, who counts Mike Tyson as a close friend, and who has a streak that caused the L.A. party girls to refer to him as “Mean Jeff Greene,” feeling vulnerable. It’s hard to think of any superrich person as vulnerable, just as it’s hard to think that a bear with outstretched claws and giant teeth is more afraid than you are. But over the past few months, it’s become clear that rich people are very, very afraid. Sometimes it feels like this was the main accomplishment of Occupy Wall Street: a whole lot of tightened sphincters. It’s not a stretch to say many residents of Park Avenue harbor vivid fears of a populist revolt like the one seen in The Dark Knight Rises, in which they cower miserably under their sideboards while ragged hordes plunder the silver.

“This is my fear, and it’s a real, legitimate fear,” Greene says, revving up the engine. “You have this huge, huge class of people who are impoverished. If we keep doing what we’re doing, we will build a class of poor people that will take over this country, and the country will not look like what it does today. It will be a different economy, rights, all that stuff will be different.”

More often than not, fears like these manifest as loathing for the current administration, as evidenced by the recent wave of Romney fund-raisers in the Hamptons. “Obama wants to take my money and give it to do-nothing animals,” one matron blurted at a recent party at the Pierre for Dick Morris’s Screwed!, the latest entry into a growing pile of socioeconomic snuff porn geared toward this audience.

Greene, a registered Democrat, isn’t buying this school of thought. “It is kind of a problem in America that so many Americans believe if they elect a different president, everything is going to be fine. This whole idea of American exceptionalism, that we’re the greatest, when people don’t have health insurance, don’t have housing,” he says, swinging past the guesthouse, which has 360-degree views of the bay, and the staff house, which does not. “There are all these people in this country who are just not participating in the American Dream at all,” he says. This makes him uncomfortable, not least because they might try to take a piece of his. “Right now, for some bizarre reason, a lot of these people are supporting Republicans who want to cut taxes on the wealthy,” he says. “At some point, if we keep doing this, their numbers are going to keep swelling, it won’t be an Obama or a Romney. It will be a ­Hollande. A Chávez.”

This past April, at the Milken Conference, the annual confab hosted by the felon turned philanthropist, Greene sat on a lunchtime panel with Charles Murray, the author of Coming Apart: The State of White America, and historian Niall Ferguson, whose recent book could have been called the same thing. “Do you see this?” Greene asked the audience, pointing to a slide that showed the widening income gap. The crowd, whose members had paid the $6,000 entry fee to get investing tips, not guilt trips, made restless noises. Then there was a smattering of impressed applause, followed by uneasy laughter. Greene blinked, surprised. “People look at Occupy Wall Street as, This is just a little kind of a disorganized joke,” he said, raising his voice. “If we take another 10 percent of middle-class America’s income, who knows what kind of other social unrest could happen in this country and the changes that could happen to our way of life?”

The level of Greene’s concern is such that, last year, he was inspired to run for Senate in Florida. But stories about the Summerwind’s goings-on dogged his campaign (“I particularly ­remember serving Tyson a vodka–and–Red Bull while he was receiving oral sex from a hired entertainer,” one former employee told the ­Broward–Palm Beach New Times), and arriving in recession-strapped towns on his private jet did little to convince most voters he understood them. Greene calls his loss “a huge mistake on behalf of the people of Florida.”

“I wasn’t as crazy as I was perceived to be,” he says in his defense. “I had a few high-profile parties. I’d be in St. Bart’s every New Year’s, big party there. St.-Tropez in July, big party every year. Malibu in May. Hundreds of people would go, so hundreds of people thought I was like that all the time.”

Greene grew up middle class, and like many in his tax bracket, he’s since lost some perspective (“Do you live in New York full time?” he asks me after I pull up in my rented Chevy Aveo). But he insists his values remain as solid as ever. “I try to think when I go to bed every night, how much good did I do in the world?” he says. During his time on the campaign trail, he told the Milken audience, he caught a glimpse of how some of the 99 percent were living, and it was chilling.

“I went to one living room in Liberty City; she was a single mom, 45 years old, 300 pounds,” he recounted. “All she was talking about was her $646 check which was coming in three days which she ­needed to buy more food.” This got the crowd’s attention, and the room filled with the sound of harrumphs. But it wasn’t this woman that set Greene’s mind on fire. It was her children. “One had been shot, one was in jail, two had gang problems.” He trailed off. “The role models those kids have—gang ­members—of course those kids are going to go in the wrong ­direction.” If the kids were given access to ­education, he continued, to ­after-school programs—

“Dream on,” Ferguson drawled in response, and the audience broke out in applause. “Dream. On.”

Greene gets this kind of reaction a lot. “Nobody gets it,” he grumbles, gunning over the boardwalk that leads from his boathouse to the beach. “I see David Koch a lot of the time. His policies are ridiculous. I don’t think he’s ever been to one of these schools where they have a rolling cart, where one computer has to go to different classrooms, and it can make so much difference, a $700 computer! I don’t think these guys realize, this is what they’re cutting off? To say to those kids, ‘Too bad, every man for himself’?”

Lately—like at a recent lunch with Steve Schwarzman, who has likened Obama to Hitler—Greene’s been trying another tactic. “Now I appeal to them selfishly,” he says. “ ‘Don’t you realize that if you don’t take care of this kid when they are 10 years old, you’ll take care of them when they are 20 and 100 instead? We just have to pay a little more taxes. It’s not going to kill us. You buy car insurance. Why not buy some democracy insurance?’ People think that Obama is this leftist, socialist guy,” he says. “But I don’t think they understand what people can go for when they are at the end of their line.”

He takes a sharp right and points out a mossy dock surrounded by toys. “Speedboat, Jet-Skis, little inflatable boat,” he recites. “Ducks!” We chug up the driveway to the main house. It’s surprisingly modest, given the property, but then people used to live differently. He and Mei-Sze plan on rebuilding as soon as they are done with their renovation in Palm Beach. He’s not sure what he wants it to look like, but one thing is likely: The new property will have gates. “You’re in Palm Beach, you’re in the Hamptons, you think you’re so secure,” Greene says. “Do you really think if you had 50,000 angry people coming across the river, you think you’re safe?”