WASHINGTON (CNN) -- An 88-year-old Maryland man with a long history of ties to white supremacist groups is the suspect in Wednesday's fatal shooting at the U.S. Holocaust Memorial Museum, two law enforcement officials told CNN.

The Web site Holy Western Empire includes writings by James von Brunn and a biography. more photos »

James W. von Brunn served six years in prison for trying to make what he called a "legal, non-violent citizens arrest" of Federal Reserve Board members in 1981 -- a prison term he blamed on "a Negro jury, Jew/Negro attorneys" and "a Jew judge," he said on his Web site, Holy Western Empire.

"He is in our files going back way into the 1980s," said Heidi Beirich, a researcher for the Southern Poverty Law Center, based in Montgomery, Alabama. "He has an extremely long history with neo-Nazis and white supremacists. He's written extremely incendiary publications, raging about Jews, blacks and the like."

Von Brunn is a native of St. Louis, Missouri, and a 1943 graduate of Washington University there. According to his online biography, he served as a Navy officer in World War II and became an advertising artist and executive after the war.

But by the late 1970s, Beirich told CNN, he had become a "hard-core neo-Nazi" and an associate of William Pierce -- the white supremacist leader whose 1978 book, "The Turner Diaries," is blamed for inspiring Oklahoma City bomber Timothy McVeigh. Watch more about von Brunn's history with neo-Nazis »

In December 1981, angered by what he called the "treacherous and unconstitutional" acts of the Federal Reserve, von Brunn entered the central bank's Washington headquarters armed with a pistol, a shotgun, a knife and a mock bomb, according to court records. He claimed to be a photographer who wanted to shoot pictures of the boardroom, and bolted up the stairs when security guards told him to wait.

While being subdued, he claimed to have planted a bomb, which forced the building's evacuation, court records state. He told officers he was upset over high interest rates -- then well into double digits -- and the state of the economy, which was in a recession.

Von Brunn, then 62, was sentenced to 11 years in prison on attempted kidnapping, second-degree burglary, assault and weapons charges. Read more about von Brunn's criminal record

In a December 1984 appeal, his court-appointed attorney, John Hogrogian, asked the court to release the defendant on bail, saying his client had "renounced his actions of December 7, 1981, and expressed remorse in a letter addressed to this court." Von Brunn has "a young son and aches to see him again," his lawyer wrote.

His trial dealt extensively with Von Brunn's "attitudes about black and Jewish people," which his lawyer said were irrelevant to the charges he was facing.

Documents were read at trial in which von Brunn wrote, "Goal: to deport all Jews and blacks from the white nations," among other racist statements. In his testimony, he also said "the basic plan was to wind up where I am now," meaning in court to raise publicity for his campaign against the Federal Reserve.

In a telephone interview with CNN, Hogrogian said that he never met von Brunn, since his client was serving his time in Ray Brook Federal Prison in upstate New York.

He said that von Brunn had approached a guard at the Federal Reserve carrying a rifle, but put his rifle on the ground and surrendered when told that he would not be allowed entry.

Hogrogian said he was surprised to hear Wednesday that his former client had shot anyone.

"I thought it was absolutely consistent with him to lay down his weapon, as he did in '81, because he viewed himself as a man of honor, military code and all that kind of stuff."

Von Brunn had told him he served as a PT boat captain during World War II, Hogrogian said.

Hogrogian, who now works for the New York City Law Department, said the two men often communicated via telephone.

"He was an intelligent guy, he was well-spoken," he said. "Didn't seem like a rabid kind of guy. He seemed sincere in his beliefs."

One of those beliefs was that the Federal Reserve was "debasing the American currency and sapping America's financial strength in the face of the communist threat, or something," Hogrogian recalled.

The appeal was based on the assertion that von Brunn's lower-court conviction should have been thrown out, Hogrogian said.

"When he was arrested in 1981, he had on his person a 12-page manifesto, which went into stuff about racial and ethnic stuff," the lawyer said. "That manifesto was read to the jury at his criminal trial, which I thought was unnecessary and inflammatory. ... There's no way he could get a fair trial reading this manifesto to the jury."

In an unpublished opinion, the three-member panel on the court of appeals disagreed and upheld the lower court's conviction of von Brunn.

Though von Brunn did not attend Hogrogian's oral argument before the D.C. Court of Appeals, several of von Brunn's supporters did -- "people who call themselves patriots, before they had militias and all that white supremacy stuff ... he called himself a patriot. That was his thing."

The lawyer said one of the "patriots" approached him in a hallway of the courthouse and told him, "I'm a friend of Jim. When we watch the football game and they play the national anthem, he stands in the room and makes everyone else stand at attention."

The appeal was denied.

Hogrogian said he could not remember whether von Brunn thanked him afterward, but did remember that von Brunn's Web site accuses him of having made mistakes in the appeal.

Asked whether he had done so, the 59-year-old responded, "Well, I was a young lawyer. I did the best I could." He added, "The D.C. Court of Appeals found no mistakes."

Von Brunn's Web site proclaims itself "a new, hard-hitting expose of the Jew conspiracy to destroy the white gene-pool." Postings attributed to von Brunn have appeared on numerous other sites, including ones that call the Holocaust and "The Diary of Anne Frank" hoaxes and bemoan the decline of the "Aryan gene pool." Watch more about von Brunn's Web site »

"Bit by bit Liberalism ascended. Bit by bit the Constitution was re-interpreted. Bit by bit government institutions and Congressmen fell into JEW hands -- then U.S. diplomacy, businesses, resources and manpower came under JEW control," one such message reads. "Whitemen sat on their collective asses and did NOTHING -- NOTHING BUT TALK."

More recently, messages attributed to von Brunn question the authenticity of President Obama's birth certificate, the subject of numerous conspiracy theories. Advocates of those theories imply that Obama was born overseas and therefore would be constitutionally ineligible to hold office.

Von Brunn was in critical condition Wednesday after being shot by a security guard at the museum. Another guard, Stephen Tyrone Johns, was killed in the shooting, museum officials said. Watch witness describe shooting »

Washington police and the FBI said the shooting appeared to have been the work of a single gunman, and there was no prior indication of the attack.

CNN's Bill Mears and Steve Turnham contributed to this report.

All About United States Holocaust Memorial Museum