Jared Kushner may have a harder time doing his job now that he’s lost access to top secret information, becoming the most high-profile victim in a White House security clearance scandal.

According to Politico, Kushner and all other White House aides operating at a high-level interim security clearance — Top Secret/SCI — were informed Friday that their clearance would be downgraded.

That’s a huge blow to Kushner, who President Donald Trump has entrusted as senior adviser with delicate national security and foreign policy projects, including placing him in charge of brokering peace in the Middle East.

“It would be very difficult for him to handle the portfolios that he has reportedly been assigned,” says Laura Terrell, clearance counsel under President George W. Bush. “For someone like Jared Kushner with a very broad remit … it’s very hard to conceive of how he could functionally do his job.”

Kushner has now lost his access to the highly classified President’s Daily Brief, according to Reuters. And experts echo that Kushner’s position in particular — overseeing a broad swath of top foreign policy issues — will be hampered by a loss of top secret clearance.

“It would be limiting if he’s doing the kind of job if you would expect of someone who is really across the whole range of White House thinking and initiatives,” says a former official in the George W. Bush Administration.

Going forward, experts say, Kushner could be kept out of meetings he may previously have attended, and there are certain documents he won’t be allowed to read. He would then also be barred from meetings or conversations concerning the contents of those documents, and will likely be kept from meetings surrounding the daily brief.

A former National Security Council official says that with Kushner’s new clearance “he will be able to review most State Department cables and some intelligence reports, but not the most sensitive intelligence information and analysis.”

In a statement this week, Chief of Staff John Kelly said the White House has “full confidence in (Kushner’s) ability to continue performing his duties in his foreign policy portfolio including overseeing our Israeli-Palestinian peace effort and serving as an integral part of our relationship with Mexico.”

The security clearance sweep comes just weeks after Rob Porter resigned as staff secretary amid allegations of domestic abuse, which exposed the fact that he’d been working in a crucial White House position for more than a year on an interim security clearance. (He denies the abuse.) In the following days, the White House repeatedly declined to clarify how many top aides were still operating on interim clearances, but Kelly ordered a crackdown last week.

“The American people deserve a White House staff that meets the highest standards and that has been carefully vetted — especially those who work closely with the president or handle sensitive national security information,” Kelly said in a memo. “We should — and in the future, must — do better.”

Trump, who has kept Kelly as his chief of staff despite recent rumors that he was considering firing him, placed the decision on Kushner squarely at Kelly’s feet.

“That will be up to General Kelly,” Trump said at the White House on Friday of Kushner’s clearance. “General Kelly respects Jared a lot and General Kelly will make that call.”

Personal dynamics between the president, his chief of staff and his son-in-law aside, having a reduced clearance threatens to undermine Kushner’s power in the West Wing, given the sensitive nature of his portfolio.

“It will severely degrade his ability to perform high-level and sensitive national security tasks,” says national security lawyer Bradley Moss. “His work portfolio will have to be reined in significantly.”

And in another setback for Kushner, Axios reported Tuesday that his spokesman Josh Raffel is leaving the White House. “Josh’s guidance was invaluable,” Ivanka Trump told Axios. “The White House won’t be the same without him.”

But Trump famously relies on family and loyalty above all else, and Kushner and his wife Ivanka occupy a rarified position in the patriarch’s inner circle of trusted confidantes. On a staffing level, White House aides are insisting that things will be the same where Kushner is concerned.

“He’s a valued member of the team,” spokeswoman Sarah Sanders said Tuesday, “and he will continue to do the important work that he’s been doing since he started in the Administration.”

Get our Politics Newsletter. The headlines out of Washington never seem to slow. Subscribe to The D.C. Brief to make sense of what matters most. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Write to Tessa Berenson at tessa.berenson@time.com.