PHOENIX -- Stung by sarcasm that deems his team amateurish, buffeted by reports of a dysfunctional organization, confused by criticism that catches him by surprise, Jimmy Haslam still has one word to describe his feelings about owning the Cleveland Browns.

"A privilege," he said as he sat next to his wife and fellow owner Dee at the recently concluded NFL owners meetings. "Unequivocally, yes, it's a real privilege to be in this position."

He said that's especially true in Cleveland, a city some 500 miles from his home in Knoxville, Tennessee.

"Besides the fact that it's really cold in the winter," Haslam said with a smile, "[Cleveland's] a really nice place to live. With great people and tremendous football fans."

Haslam stands firm in his commitment, but he has had his share of tumult and upheaval in less than three years of owning the Browns. From general manager Ray Farmer being suspended four games because of illegal in-game texting, to offensive coordinator Kyle Shanahan leaving with two years left on his contract, to first-round pick and expected face-of-the-franchise quarterback Johnny Manziel finishing rehab, to former All-Pro receiver Josh Gordon facing another suspension for substance abuse, headlines have not been kind to the Browns. And that was all within the past few months.

"Let's face it," he said, "it hasn't been the smoothest start in the world."

Since the family bought the team for just more than $1 billion in October 2012, the Browns have been through three head coaches, three general managers, seven quarterbacks and (in two full seasons under Haslam) 21 losses in 32 games.

In the same span, Pilot Flying J, the Haslam family's company, went through an FBI raid on its headquarters as part of an investigation into alleged fraud regarding fuel rebates. Several executives have plead guilty, and Haslam had to pay heavily to fix the mess -- in fines, restitution and in his image in Cleveland.

"Nobody likes criticism, let's face it," Haslam said. "We've had our share, some of it probably self-inflicted and deserved. Others maybe not.

"But it is what it is."

Johnny Manziel was expected to become the face of the team but had an up-and-down rookie season. Bob Donnan/USA TODAY Sports

Last week Haslam even had to deny a story about him switching ownership of teams, a scenario that would see him giving up the Browns for the Tennessee Titans. He blistered the report, saying there was no foundation and expressing anger that he even had to address it.

"We feel really fortunate to own an NFL team, we feel really fortunate that it's Cleveland, and we're totally committed to turning the franchise around and making it a consistent winner," he said. "We're committed to do that, and it's not easy. OK?

"It's not easy."

The final two words say much. Haslam joined the Browns after serving as minority partner with the Steelers, a team and organization that exudes stability and success. The Steelers have had the same number of coaches in 45 seasons as the Browns have had in three. But Haslam has learned that just building his team to the point of stability can be undone with pratfalls and missteps. Do it in an environment of negativity like the one in which the Browns reside, and the team rarely gets the benefit of the doubt.

"Listen," Haslam said, "all organizations feel like they're picked on and that you guys [in the media] are too tough on us. There's times we say that, but I just say, 'There's no sense spending time [on it]; they're going to write what they want to write.'

"The best thing we can do is win and build up some equity and capital. Until we win and win consistently, we don't have that."

Browns coach Mike Pettine calls Haslam an involved, interested and active owner, but not a meddling one.

"It's upsetting to see him get maligned the way he has been when he's been nothing but supportive," Pettine said during the owners meetings. "He wants to know the why. That's one of the reasons why he's gotten to where he is in business. He's very inquisitive."

The Haslams' principal residence remains in Knoxville, but they have a home in Bratenahl, a village east of Cleveland where the homes sit on Lake Erie and income ranks sixth in Ohio. His Knoxville residence, a historic home built in 1928, sits on the banks of the Tennessee River. It recently was listed for sale with an asking price of $4.9 million (the Haslams purchased it in 1996 for $1.4 million).

"We've adopted [Cleveland] as our town," Dee Haslam said. "But we haven't been adopted by the town yet. We have to kind of prove ourselves, and we're willing to be patient and humbly ask for the blessing of the community that we can get involved and make a difference."

The pair make clear they couldn't ask for a warmer welcome. They prefer to move in slowly, and assume they will be assimilated over time.

"Everybody has been great to us," Dee Haslam said.

She spoke as a member of the Browns ownership group, a role she has assumed more and more. Dee is one of two women on the NFL's conduct committee. She also is Executive Producer and CEO of RIVR Media, a film production company responsible for documentaries and series like "Trading Spaces" and "Renovation Realities."

"Jimmy is obviously the primary owner and face of the team," Dee Haslam said. "But I'm very involved."

She said she doesn't ask or need permission from her husband to act, that she's an independent and active participant. She's passionate about addressing domestic violence, and active in the team's broadcast interests. She also took a strong stand against the Browns being featured on "Hard Knocks" this year.

"It's a little sensitive from a female that's been around business all her life to be referred to as a spouse," she said. "I got on the conduct committee not because I was Jimmy's spouse; I think I got on the conduct committee because they felt I added value for who I am."

As president of the Cleveland Browns Foundation, Dee runs the team's charitable arm. She also recently was behind a $1 million donation to Cleveland's Breakthrough Schools from the The Haslam 3 Foundation, the couple's private charity.

"I'm fine being referred to as spouse," Dee Haslam said. "I'm very proud of that. But I'm also an owner of the team."

Jimmy Haslam has not been afraid to bring change to the Browns beyond the obvious on-field and front-office overhauls. In just three years he has allowed long-standing contracts with the team's broadcast (radio and TV), insurance and medical care partners to run out, then signed deals with new companies. It took him very little time to sell naming rights to Cleveland Browns Stadium, something the Lerner family resisted. This offseason, he raised ticket prices -- by as much as 30 percent.

The drive for revenues has not gone unnoticed; team president Alec Scheiner said recently that the business end of the Browns had a good year. Haslam also has spent upward of $130 million as part of a renovation of FirstEnergy Stadium that drew positive reviews.

A huge change will come April 14 as the Haslams unveil new uniforms for a team whose look has not been radically changed since it joined the NFL in 1950. Finding the combination of new while maintaining tradition could not have been easy.

"That was the goal, to balance it well," Dee Haslam said. "I think we've done that."

The uniform unveiling will take place at the Cleveland Convention Center, on a one-hour television show replete with an "orange carpet" where media can interview "dignitaries" as they head inside.

The Haslams took a lot of barbs when a much-hyped new logo turned out to be a slight variation of the orange on the helmet. The Cincinnati Bengals and renowned Cleveland Orchestra even poked fun at the Browns on April Fool's Day with new logos that barely changed or did not change at all. A one-hour TV show on uniforms looms, and it seems there may be trepidation about the next reaction.

"Look, there's going to be a zillion opinions on whether [people] like [the uniform] or don't like it or whatever," Dee Haslam said. "It's really a hard call, and at the end of the day it's how you play the game."

Playing the game has been the Browns' biggest issue since well before Haslam took over. The follies have continued under his watch.

Still, Haslam seems at times flabbergasted by some of the questions sent his way and some of the things written about his team. He says he understands the criticism, but obviously pays attention and recoils at some of it. He knows his team has been mocked, but he sees his organization as one that is cohesive, dedicated and hard-working.

In October 2016 he will make a final $300 million payment to former owner Randy Lerner that will complete his purchase. He wants to spend more time in Cleveland and "at a certain point in time" live there full-time. He embraces the team's history, and when asked about a Paul Brown statue outside the stadium he readily admits he hasn't thought about it but he should.

When it's mentioned he's been through some rough water, he says "yeah" and admits the criticism "caught us off guard."

"I just accept the fact that we have to win," he said, adding: "If we don't win in four or five years, it's our fault."