Washington (CNN) The Justice Department plans to take dozens of cases away from an immigration judge who has delayed deportation orders, in part for perceived criticism of Attorney General Jeff Sessions, the union representing immigration judges said Wednesday.

CNN reported Tuesday that the Justice Department replaced Philadelphia Immigration Judge Steven Morley with an assistant chief immigration judge last month to hear a single case on his docket, which resulted in a young undocumented immigrant, Reynaldo Castro-Tum, being ordered deported.

Assistant Chief Immigration Judge Jack Weil told Morley that comments in the Castro-Tum case were perceived as "criticism" of the Board of Immigration Appeals and attorney general's decisions and that they were "unprofessional," according to the grievance filed by the National Association of Immigration Judges. The cases all involve young undocumented immigrants and whether they got adequate notice from the government about hearings at which they failed to appear. Weil also told Morley that he himself should have either ordered Castro-Tum deported or terminated the case altogether.

It's the most public fight yet between the union that represents the nation's roughly 350 immigration judges and Sessions, who has intently focused on the immigration courts under his purview. The immigration judges have long bemoaned their structure under the Justice Department, but have taken particular issue with many of the moves pursued by the Trump administration that they say interfere with their ability to conduct fair and impartial court proceedings.

Unlike federal judges, immigration judges are employees of the Justice Department and the attorney general has the authority to hire them, manage their performance measures and even rule on cases with binding authority over how the judges must decide similar issues.

Read More