If your investment portfolio includes oil stocks, it is time to sell some of them, CNBC's Jim Cramer said on Thursday.

"I would scale them back," the "Mad Money" host advised.

If oil industry stocks "didn't have a sustained move" this week after roughly half of Saudi Arabia's oil production was lost in a drone attack, then "I don't know when it will," Cramer said.

The attack briefly caused the price of crude oil to rise to the mid-$60s, Cramer noted, but the U.S. West Texas Intermediate crude is back to around $58 a barrel, "as if there had not been an attack at all."

There are multiple, powerful reasons why the oil complex faces difficulties, Cramer said.

Among them: a slowing economy, a robust supply of crude in West Texas' Permian Basin and environmental concerns.

The increase in production in West Texas explains why the oil market wasn't impacted long-term by the attack in Saudi Arabia, Cramer said.

Plus, that production of oil and gas got a lift by the completion of a new pipeline, Cramer said. Now, it is "pushing American production up from 12 million barrels a day to 17 million in the not too distant future. A decade ago it was at 5 million," Cramer said.

"The market simply can't handle that much new supply, which is why oil in the outyears is so darned cheap," Cramer said.

The environmental concerns are important to consider, too.

Cramer pointed to the ban on natural gas in new residential projects that San Jose, California, passed this week as the latest warning sign for fossil fuels.

"This news is huge," said Cramer, who noted San Jose is the largest U.S. city to approve this kind of restriction. "It tells us that the future is now for renewable fuels."

While older money managers may hope the traditional energy complex will make a comeback, Cramer said the mindset is different among younger investors.

"Younger portfolio managers believe that oil and gas are the new coal," Cramer said, adding: "If you take climate change seriously, and most younger people do, including younger portfolio managers, then it's hard to believe in the long-term health of this industry."

Cramer said he is heeding his own advice, explaining that his charitable trust "took a brutal loss this week" on Schlumberger, the Houston-based supplier of technology for the oil and gas industries.

Its sock is closed at $37.76 on Thursday, down from its 52-week high of $63.65. In September 2014, it was trading north of $100.

"I can't believe we sold this stock so much lower than where it was trading when oil was at half of these levels," Cramer said. "But after a major supply shock failed to move the forward curve, we simply can't expect the oil companies to be aggressive long-term."