Washington (CNN) The top official at the New York prison that had housed Jeffrey Epstein before his apparent suicide is being moved temporarily as the FBI and the Justice Department's inspector general investigate the circumstances of the death.

Two employees at the Metropolitan Correctional Center who had been assigned to Epstein's unit are also being placed on administrative leave, Justice Department spokeswoman Kerri Kupec said, adding that "additional actions may be taken as the circumstances warrant."

The moves, which were directed by Attorney General William Barr, come one day after Barr said there were "serious irregularities" at the New York prison and that he was "appalled" and "angry" to learn of the facility's "failure to adequately secure this prisoner."

The warden, Lamine N'diaye, will be moved to a Bureau of Prisons regional office and will be replaced by James Petrucci, the warden of the federal prison in Otisville, New York

A picture of dysfunction in the unit where Epstein was held as he faced new charges of sex trafficking has emerged since his death over the weekend.

Read More