Washington (CNN) -- Pakistani intelligence secretly funneled at least $4 million to a Washington front group whose leaders improperly lobbied U.S. officials over the disputed territory of Kashmir, federal agents alleged Tuesday.

A Pakistani-American man who served as director of the Kashmiri American Council is in federal custody, while a second man accused of steering money to the organization is believed to be in Pakistan, the Justice Department said. The KAC director, Syed Ghulam Nabi Fai, "acted at the direction and with the financial support of the government of Pakistan for more than 20 years," an FBI arrest affidavit states.

One U.S. congressman quickly gave $4,000 donated by the two men charged in the case to charity, while another said he would consider a similar move if the source of the money was in question.

Fai and his co-defendant, Zaheer Ahmad, have been charged with conspiring to violate the Foreign Agents Registration Act, which requires lobbyists acting on behalf of another nation to register with the U.S. government. The charge carries a possible prison term of up to five years.

Short, balding and bespectacled, Fai made a brief initial appearance in federal court in the Washington suburb of Alexandria, Virginia, Tuesday afternoon. He wore a short-sleeved, checked shirt and dark slacks at the hearing, during which no plea was entered.

U.S. Magistrate Judge Theresa Buchanan urged Fai to get a local attorney to defend him after the defendant told her his lawyer was in Rochester, New York. Buchanan set a detention and preliminary hearing for Thursday at afternoon.

The goal of the operation was to tilt U.S. policy in Pakistan's favor on Kashmir, a Himalayan territory disputed between India and Pakistan since Pakistan's independence from Britain in 1947. Both nations claim the entire region, which has been divided between them by a U.N.-monitored cease-fire line. The dispute has been the flashpoint in two wars between the nuclear-armed south Asian rivals.

Fai was operating as a Pakistani agent as early as 1995, the FBI affidavit states. His group organized seminars, lectures and conferences on Kashmir, and Fai attended a dinner with members of Congress in 2008, according to the affidavit.

Another longtime FBI source said up to 80 percent of Fai's public statements are provided by Pakistan's powerful Inter-Services Intelligence agency "to repeat and disseminate verbatim," the document states. "The other 20 percent of the KAC's messaging consists of Fai's own ideas, which have been pre-approved by the ISI but not provided by them."

According to the FBI affidavit, federal agents learned of the scheme from an unidentified informant who faced a U.S. jail term. The man told the FBI he had helped conceal the source of KAC funds, which he said originated with the ISI.

The complaint states that Ahmad lined up donors within the United States to contribute to the KAC, "in return for repayment of those amounts in Pakistan." The informant said the money was transferred to Fai's organization via Ahmad, an American citizen living in Pakistan.

Pakistan has been a U.S. ally in the battle against the al Qaeda terrorist network and the Taliban in neighboring Afghanistan. But relations between Islamabad and Washington have been strained by the American commando raid that killed al Qaeda leader Osama bin Laden in Pakistan and other issues.

Federal campaign contribution records show Fai and Ahmad, also a U.S. citizen, contributed to a number of members of Congress between 1998 and 2008. The charges specifically link Pakistani funds to two 2004 contributions -- one by Ahmad and one by his nephew as a "straw donor" -- to members of Congress identified only as elected officials A and B.

Based on FEC records, "Elected Official A" appears to be GOP Rep. Joe Pitts of Pennsylvania, who received a $2,000 contribution from Ahmad on the date mentioned in the affidavit. He got another $2,000 from Fai on the same day in September 2004, those records show.

Pitts spokesman Andrew Wimer told CNN that the eight-term congressman has made $2,000 donations to two non-profits in his Lancaster-area district, the Water Street Mission and the Boys and Girls Club, since news of the charges broke.

Fai, meanwhile, made personal contributions of $23,500, mostly to Republicans, who controlled Congress for most of the period. He gave $7,500 to GOP Rep. Dan Burton of Indiana, a self-proclaimed "champion of Pakistan," between 1997 and 2010, FEC records show. In a statement posted on his congressional Web site, Burton said he was "deeply shocked" by Fai's arrest.

"For as long as I've known him, Dr. Fai has been either a permanent legal resident of the United States or a citizen and as such, any political contributions I may have received from Dr. Fai over the years are completely legal," Burton said. He said that in the 20 years he has known Fai, "I had no inkling of his involvement with any foreign intelligence operation and had presumed our correspondence was legitimate."

"However," he added, "if there is any doubt about the origin of these contributions, I will donate those funds to the Boy Scouts of America."

The biggest chunk of Fai's contributions went to the National Republican Senatorial Committee, to which he gave $9,500 between 2004 and 2008, according to the Federal Election Commission. Fai also contributed $250 sums to two Democratic presidential campaigns, former Vice President Al Gore's 2000 effort and President Barack Obama's 2008 bid.

The Justice Department said there was no evidence that any elected official who received contributions from Fai or the Kashmiri American Council knew the money "originated from any part of the Pakistani government."

CNN's Carol Cratty, Matt Smith and Katie Glaeser contributed to this report.