OAKLAND — When it comes to the National Football League returning to Los Angeles, the elephant in the room is impossible to ignore.

With a tip of the cap to legendary broadcaster John Facenda, the elephant’s face is weather-beaten, he wears a hooded sash with a silver hat about his head and a bristling black mustache.

And he scares folks old enough to remember the last time the Raiders called Los Angeles home and how their games at the L.A. Memorial Coliseum sometimes became sideshows to the shenanigans going on in the stands.

The Raiders, back in Los Angeles?

Should we start hiding the women and children?

Would the NFL even contemplate going down that path again?

As far as the NFL holding the past against the Raiders in a possible move to Los Angeles, it’s not as big an issue as some might think.

Or, as a high-ranking NFL executive recently reminded me: There is no elephant. And by the way, there were other rowdy stadiums back then in the NFL. It’s a different time now. Stadiums have more control and more scrutiny.

The Raiders and San Diego Chargers have joined forces to build a stadium in Carson pending fading stadium pushes in their local markets. Meanwhile, St. Louis Rams owner Stan Kroenke is pushing a stadium project 14 miles away in Inglewood. The NFL could decide by the end of 2015 which teams relocate to Los Angeles and where they will play.

Yet in Los Angeles, it’s hard to shake the memories of how the Coliseum was turned into a nasty playground by a rowdy element of Raiders fans more interested in creating havoc than enjoying a football game.

Those of us old enough to remember can recall the fights that broke out in the stands. Or how a Pittsburgh Steelers fan was critically injured in 1990 after getting viciously attacked as he walked through a crowd of jeering Raiders fans. That beating led to a one-game beer ban and sales restrictions for later games at the Coliseum.

And we all remember how the Raiders’ color scheme and image were adopted and embraced by local gangs, creating an ugly, ominous connection the Raiders never seemed to acknowledge, let alone distance themselves from.

It would be less than honest to pretend like that never happened, or that there aren’t concerns a repeat might happen if the Raiders end up relocating to Los Angeles.

In fact, after spending time recently in Oakland, it’s clear the Raiders feel the issue is a key hurdle they need to clear to ensure a successful landing back in Southern California.

The Raiders declined comment, but there is strong conviction from within the organization that the Raiders brand is as iconic as any in professional sports, and certainly strong enough to stand tall in the competitive Los Angeles market. Provided, of course, they field a quality football team.

The image issue, however, needs to be addressed.

The Raiders are aware of the old scouting report and will stop at nothing to change it.

In fact, they’ve already begun the process in Oakland and insist they’ll continue whether they remain in the Bay Area or move to Los Angeles.

Over the past few years, a seismic shift has occurred in an organization that once promoted its renegade image so fervently it almost seemed to be inviting the riff raff rather than discouraging it.

The Raiders now have a fan Code of Conduct they enforce with diligence at O.co Coliseum. Over the past two seasons, dozens of season-ticket holders have been banned after fellow fans reported them for poor behavior.

“Season-ticket holders have been finding out if you or anyone else with season tickets fights, your tickets will be revoked,” said Ray Perez, a 27-year-old season-ticket holder from Sacramento.

Back in the day, that never happened.

Meanwhile, the police and security presence has grown significantly at games, just as it has at venues across the league.

And at a stadium still known for the ominous Black Hole, where some passionate fans are decked out like Halloween, a kids zone is also now in place.

“The Raiders have done a great job working with the fan base, law enforcement and the Coliseum officials on clamping down on unruly fans and overall hostile activity in general,” said longtime season-ticket holder Edward Toomey, who lives in South San Francisco. “The Raiders have brought me and other fans to provide and offer solutions on how to improve the overall experience with a safe experience being the top priority.”

Make no mistake, the Raiders fully embrace the fans they attract from all the various corners in which they reside. In fact, it’s a source of pride. And the hope, like all teams, is to create an intimidating atmosphere for opposing teams.

But there are lines they no longer tolerate being crossed. And while the Raiders can’t always control who comes into a stadium, they are determined to get rid of anyone who steps out of line once they gain entrance.

“I thinks the culture has slowly changed,” Perez said. “There are a few idiots who are against it, but security has been more keen to protecting opposing fans, and Raider fans in the Black Hole have been more accepting of opposing fans compared to years past.”

It’s a process the Raiders insist will continue if they move south, and one they’ve already stressed with Carson leaders.

“And I’m 100 percent confident that the Raiders are fully committed to creating a fan-friendly, family-friendly environment,” Carson Mayor Al Robles said. “I don’t just believe that based on what the Raiders are committed to doing and while working in conjunction with our sheriffs, I also believe we live in different times.

“What happened at the (Los Angeles) Coliseum in the ’80s and ’90s, we’re talking 20, 30 years ago. It’s a completely different world now compared to then. There is much more awareness to creating a fan-friendly environment. And I know the Raiders, as are we, are committed to creating one of the safest, if not the safest, stadium experiences in the country. And I know that’s exactly what we will do.”