Washington (CNN) Judge Steven Morley has overseen the immigration case of Reynaldo Castro-Tum for years. But last month when Castro-Tum was officially ordered deported, it wasn't Morley at the bench.

Instead, the Justice Department sent an assistant chief immigration judge from Washington to replace Morley for exactly one hearing: the one that ended Castro-Tum's bid to stay in the US.

The unusual use of a chief immigration judge from headquarters has raised concerns from retired immigration judges, lawyers and the union for active immigration judges. They say the move seems to jeopardize the right to a fair process in immigration courts.

It also highlights the unique structure of the immigration courts, which are entirely run by the Justice Department, and the ways that Attorney General Jeff Sessions -- who serves as a one-man Supreme Court in these cases -- has sought to test the limits of his authority over them.

The saga of Castro-Tum starts in 2014, when he crossed the border illegally as a 17-year-old. The Guatemalan teen was apprehended by the Border Patrol, which referred him to custody with the Health and Human Services Department as an unaccompanied minor. He was released to his brother-in-law a few months later and registered his brother-in-law's address with the government. Multiple notices of court hearings were sent to that address, the government said.

Read More