EVANSVILLE, Ind. — President Donald Trump vowed Thursday to "get involved" if his administration's Justice Department and FBI don't start "doing their job and doing it right."

His remarks, delivered at a campaign rally, came just hours after he told Bloomberg News that Attorney General Jeff Sessions would remain in his job at least until the midterm elections but declined to say whether that would be true after voters go to the polls.

"All I can say is our Justice Department and our FBI, at the top of each because inside they have incredible people, our Justice Department and our FBI have to start doing their job and doing it right," the president said here before thousands of fired-up supporters. "What's happening is a disgrace and at some point — I wanted to stay out — but at some point if it doesn’t straighten out properly ... I will get involved."

At one point, Trump had to pause his remarks for several minutes because of a protester, whom the crowd jeered; she responded with her middle finger. "Where the hell did she come from?" Trump asked. It was one of the president's most enthusiastic crowds, drawing people who had, in some cases, driven several hours and across state lines to see him. Chants of "U-S-A!" and "Lock Her Up!" and the like boomed through the downtown Ford Center.

Trump has long been angry about Sessions' decision to recuse himself from special counsel Robert Mueller's probe into the Trump campaign's ties to Russia and possible obstruction of justice, and he has become increasingly vocal in articulating what he says was a "deep state" plot to undermine his campaign and presidency.

Frequently, he laments that the Justice Department is not going after his 2016 rival, Hillary Clinton.

And it was Clinton who provided the lead-in to Trump's promise to "get involved" if the Justice Department doesn't dance more to his beat. Complaining that the media didn't give her enough scrutiny, Trump said, "You can have the biggest story about Hillary Clinton, I mean look at what she's getting away with and let’s see if she gets away with it."

The crowd chanted "lock her up!" — a staple of Trump campaign rallies that suggests Clinton should be imprisoned even though she hasn't been charged with anything.

That's when Trump went after the Justice Department and FBI.

Earlier in the day, in an Oval Office interview with Bloomberg, he said that he views Mueller's probe as an "illegal investigation," citing "great scholars" who say it should never have been launched.

Some Republicans have worried that a Justice Department bloodbath could harm their chances of holding the House and Senate in the midterms, but Trump told Bloomberg he wouldn’t move on the attorney general before then.

"I just would love to have him do a great job," Trump said. But he would not say whether Sessions would stay in place after the election.

It is not unheard of for a president to fire an unpopular Cabinet official in the wake of a midterm election. President George W. Bush famously canned Defense Secretary Donald Rumsfeld right after Democrats won the House and Senate in the 2006 midterms.

Sen. Lindsey Graham, R-S.C., who has formed a close bond with Trump, said earlier this week on "Today" that the relationship between the president and his attorney general is “beyond repair.”

Immigration was a key feature of Trump's speech here Thursday night, as he told the crowd that "this election is about safety and this election is about jobs."

As he often does, Trump portrayed Democrats as weak on border security and the violent gang MS-13.

"We want to abolish MS-13, they want to abolish ICE," he said.

Trump was in Indiana to boost the hopes of Republican candidates before the midterms. The biggest race on the ballot here is Republican Mike Braun's effort to unseat Sen. Joe Donnelly.

Donnelly is one of a handful of highly vulnerable Democrats in the country, as Trump won 56.5 percent to 37.5 percent over Clinton in Indiana in 2016. His numbers in Evansville’s Vanderburgh County were just slightly less impressive, at 55.2 percent to 38.9 percent.

Marjorie Hershey, a political science professor at Indiana University, said Trump was not really trying to win over voters at the rally.

"Trump is not one to try and expand the Republican base," she said. "The intention of his visit is not to increase Republican support. Rather, Trump is all about mass adoration and being in front of his own base."

President Donald Trump gestures during a campaign rally at Ford Center in Evansville, Indiana, on Thursday. Mandel Ngan / AFP - Getty Images

Trump's popularity has ebbed here, according to polling from Morning Consult. In January 2017, 55 percent of Hoosiers approved of Trump while 33 percent disapproved. In July, his approval rating had slipped to 51 percent and his disapproval was up to 45 percent.

That's in part because of his tariff policies, according to Robert Dion, chair of the political science department at the University of Evansville.

"We are the biggest steel-producing state in the country, and there’s also a lot of agriculture in the state," Dion said. "A lot of businesses across the state are deeply involved in the international economy. So when you start engaging in trade war and tariffs, it has immediate and negative consequences. You can hear the grumbling across the Midwest."

Trump bashed Donnelly for voting against tax cuts, efforts to repeal Obamacare and other items on his agenda.

"We're always happy to have President Trump in Indiana," Donnelly said in a statement released after the rally, "but Hoosiers still want a senator who always puts them first before any politician or political party."

Jonathan Allen reported from Evansville. Lauren Egan contributed from Washington, D.C.