WASHINGTON — When the conservative financier Charles Koch sent out invitations for a political retreat in Palm Springs later this month, he highlighted past appearances at the gathering of “notable leaders” like Justices Antonin Scalia and Clarence Thomas of the Supreme Court.

A leading liberal group is now trying to use that connection to argue that Mr. Scalia and Mr. Thomas should disqualify themselves from hearing campaign finance cases because they may be biased toward Mr. Koch, a billionaire who has been a major player in financing conservative causes.

The group, Common Cause, filed a petition with the Justice Department on Wednesday asking it to investigate whether Justices Antonin Scalia and Clarence Thomas should have recused themselves in the case, involving Citizens United, because of their attendance at past retreats organized by the conservative financier Charles Koch, whose company operates a foundation that is a major contributor to political advocacy groups.

Common Cause also cited the role of Mr. Thomas’s wife, Virginia Thomas, in forming a conservative political group opposed to the Obama administration as grounds for his disqualification.