Two men found guilty of terrorism charges after being accused of plotting to derail a passenger train are asking Ontario's highest court to grant them a new trial, arguing the jury that convicted them was improperly selected.

Lawyers for Raed Jaser argue the judge who oversaw the case made several errors, including in rejecting their client's request as to the method of jury selection.

Jaser's co-accused, Chiheb Esseghaier, continues to be self-represented but a lawyer appointed to assist him through the legal process agrees the jury was not properly constituted and says a new trial must be ordered for both men.

Though Jaser has raised several other grounds of appeal, the court has said it will first weigh the issue of jury selection before considering the others. The challenge is expected to be heard in a Toronto court on Tuesday.

Jaser and Esseghaier were found guilty in 2015 on a total of eight terror-related charges between them. They were sentenced to life in prison with no chance of parole until 2023.

The Toronto judge who presided over their trial found both men had not renounced their extremists beliefs, nor had they accepted responsibility for their offences.

Esseghaier, a devoutly religious Muslim, often interrupted court proceedings with rambling speeches and at times prayed in the prisoner's dock. His mental state was raised during the sentencing phase of the trial.

In documents filed ahead of Tuesday's hearing, Jaser's lawyers said the law regarding jury selection was in flux during the trial, but over time the interpretation used by the trial judge has been found to be wrong.

During the selection process, prospective jurors may face questions regarding their ability to remain impartial. Two people from the jury pool take on the role of "triers," meaning they consider the answer and determine whether there is sign of bias. Lawyers for both the Crown and the defence then decide whether to allow the person on the jury.

Each newly appointed juror replaces one of the two triers so that the responsibility is shared -- a process called "rotating triers."

Though it was not stated in the Criminal Code, common law granted the court the discretion to exclude from the room all other prospective jurors or both prospective jurors and those already selected who were not currently acting as triers.

In 2008, an alternative process was created in the Criminal Code allowing the accused to request to have prospective and all selected jurors excluded during the questioning, provided they could show it was necessary to preserve the jury's impartiality.

If the court granted the request, it would then appoint two people to assess all the prospective juror responses. These are called "static triers" and do not get to serve on the jury.

At the time when jurors were selected for Jaser and Esseghaier, there was debate over the amendment and whether it eliminated judges' discretion to exclude prospective jurors and some selected jurors when relying on rotating triers, the appeal submissions said.

But since then, the appeal court has found judges retain that discretion, Jaser's lawyers said.

As a result, the trial judge erred in denying Jaser's request to have the jury chosen by rotating triers with prospective jurors kept out of the room, they argued.

The judge ruled he did not have the discretion to grant the request and said since Jaser wanted to exclude prospective jurors to ensure impartiality, the only route available was through static triers, the lawyers said in the documents.

Jaser, however, had not formally applied for static triers, and without that application, the judge had no jurisdiction to impose that method, the lawyers said.

"The trial judge found that an exclusion order was necessary but wrongly declined to exercise his common law jurisdiction to make one. Had he applied the law correctly, he would have granted the order Jaser requested," the documents said.

The issue is particularly important in a case with multiple defendants, they said, noting Esseghaier's failure to apply for static triers should have bolstered Jaser's request.

The lawyer assisting Esseghaier said in documents filed for the appeal that the trial judge should have sought to balance the interests of both men rather than impose a process he considered best.

"No doubt trials involving self-represented accused persons are difficult trials for judges to preside over. This trial was made even more difficult by the fact that Esseghaier at the time did not recognize Canadian law and wanted to be tried under the Qur'an," the lawyer wrote.

"This difficulty, however, does not justify trial judges failing to advert to the statutory rights of self-represented accused persons, or treating their rights as any less important than the rights of accused persons represented by counsel."

Both men also previously indicated they wished to challenge their sentences, which Jaser argued in his notice of appeal was "excessive, harsh and unfit in all the circumstances."