President Donald Trump has asked chief of staff John Kelly for help in pushing his daughter Ivanka Trump and son-in-law Jared Kushner out of their official White House positions even as he encourages the two to remain as aides, the New York Times reported Thursday night, citing unnamed White House aides.

Trump has also said that his daughter and son-in-law should have never come to the White House, according to the New York Times.

The President has grown particularly frustrated with Kushner recently, given the news that his security clearance was downgraded and the uptick in scrutiny of the Kushner family business, two people familiar with Trump’s thinking told the Times.

Read the New York Times’ full report here.