MIAMI -- If LeBron James ever wanted Erik Spoelstra fired as coach while a member of the Heat, he never expressed that sentiment to team president Pat Riley.

"Not from him to me, ever," Riley said Thursday of James, who left the Heat two seasons ago in free agency to return to Cleveland. "So a lot of stuff is following him out the door. Whether it's right or wrong, it's just the nature of it. But as far as that goes, no, he never, ever walked in and said anything."

Riley's comments come on the heels of Israel-based reports earlier this week in which Heat limited ownership partner Raanan Katz suggested James was largely responsible for David Blatt's firing in Cleveland and said James pushed Riley to dismiss Spoelstra while with Miami.

Pat Riley said rumors about LeBron James' supposed displeasure with coach Erik Spoelstra are among "a lot of stuff" following him after his exit from Miami. Jesse D. Garrabrant/NBAE/Getty Images

Katz, who owns less than 10 percent of the Heat, has since told multiple media outlets that his comments in the interview with an Israeli radio station were taken out of context. James also denied that he had any meaningful dealings with Katz, who sits courtside at most Heat home games and routinely interacts with visiting players and coaches at AmericanAirlines Arena.

Aside from Katz's courtside access during games and attendance at season-ticket holder events, multiple team sources told ESPN.com that he has essentially no role in the Heat's basketball operations or personnel decisions.

James left Cleveland as a free agent to sign with the Heat in 2010 and led Miami to four consecutive NBA Finals, including consecutive championships in 2012 and 2013, before opting out of his contract to become a free agent again in 2014.

Big Game Alert! The Spurs head to Cleveland to take on the Cavs, ESPN on ABC (1/30). Watch Live on ESPN!

During James' first season in Miami, there were some early internal conflicts among players and Spoelstra as the Heat stumbled to a 9-8 start. At the time, there was widespread speculation that James was not pleased with some of Spoelstra's coaching methods. But Riley consistently backed Spoelstra throughout the turbulent times in that first season.

Riley told reporters Thursday that some of the rumors that continue to surface from James' time in Miami remind him of what he dealt with amid his own controversial moves, from coaching the Showtime Lakers in the 1980s to the Knicks in the 1990s and ultimately the Heat.

"I've been in that situation twice," said Riley, who was presented with the Joe DiMaggio American Icon Award on Thursday. "I left Los Angeles, and a lot of stuff followed me out the door to New York. A lot of stuff followed me down here to Miami. It took almost 10 years to sort of get out of that storyline."

Riley was caught off guard by James' decision to return to Cleveland and has spent the past two seasons publicly setting the Cavaliers as the primary target for the Heat to compete with in the Eastern Conference. After struggling through an injury-ravaged 37-45 season a year ago, the Heat are 25-21 heading into Friday's game against Milwaukee.

"I think we're right in the middle of the pack," Riley said. "It seems as though there's one team in the East that might be, not head and shoulders, but better than everybody, that being Cleveland. But I think we're going to be right in the thick of things."