Washington (CNN) The U.S. government thinks Jakiw Palij was a guard at a Nazi concentration camp, but the 92-year-old is quietly living out his last years not in prison -- confined by old age to a second-story apartment in a modest red-brick duplex in one of the most diverse sections of New York City.

Jackson Heights, Queens is an evolving, largely immigrant neighborhood a short 20 minute subway ride to midtown Manhattan.

Palij's life there represents the closing of an era: the only remaining active case from the Nazi era pursued by the Justice Department's Office of Human Rights and Special Prosecutions.

Despite more than a decade of deportation attempts by the Justice Department, Palij, who worked at the Trawniki concentration camp in Poland, likely will die here.

A federal judge ordered Palij deported in 2004, but none of three European countries to which he could be sent will take him.

In court filings, he has denied wrongdoing, claiming that he and other young men in his Polish hometown were coerced into working for the Nazi occupiers.

JUST WATCHED Reporter's Notebook: Finding a Nazi in Queens Replay More Videos ... MUST WATCH Reporter's Notebook: Finding a Nazi in Queens 02:11

Eli Rosenbaum has dedicated a career to pursuing men like Palij.

Rosenbaum was an intern when he joined what was then the Office of Special Investigations in 1979, shortly after it was created.

Now director of Human Rights Enforcement Strategy and Policy, Rosenbaum has become the Justice Department's best-known Nazi Hunter, helping the department pursue 137 cases, of which 107 were successful in stripping citizenship or deporting accused Nazis.

Among them: John Demjanjuk , who was convicted in Germany of being an accessory to more than 28,000 murders for his work as a guard at the Sobibor death camp in Nazi-occupied Poland, after the Justice Department human rights unit pursued his deportation.

Michael Kolhnhofer was so incensed at the deportation efforts of the Justice Department that when reporters showed up at his home in Kansas City in 1996 to ask questions he started shooting out his front door. The former Nazi guard at the Buchenwald death camp died after a shootout with police.

For Rosenbaum, a career as Nazi hunter was accidental.

After his internship stint he envisioned a legal career, perhaps in finance.

Growing up, his family in Long Island, N.Y., didn't talk about the Holocaust. The term "Holocaust" wasn't even common parlance until a 1970s NBC broadcast of a teleplay based on the Nuremberg Trials.

Rosenbaum recalls a snowy drive in New York decades ago with his father, who began telling him of his wartime military service. His father described arriving at Dachau, Germany a day after it was liberated. Then he stopped speaking. Rosenbaum looked over at this father. He was holding back tears.

"That was a time when you just didn't see your dad cry," Rosenbaum recalls.

The human rights office got its start after congressional and public pressure prompted the Justice Department to try to track down former Nazis who had moved to the U.S. -- some of them with the help of U.S. spy and law enforcement agencies that recruited former Nazis with technical knowledge. In recent years, the office has also investigated cases related to atrocities in Bosnia and Guatemala. There remain a handful of ongoing World War II probes and Rosenbaum also oversees cases involving more recent war atrocities.

In the Nazi cases, a handful of attorneys, historians and researchers looked into as many as 70,000 names of possible Nazis. Their obstacles were many: they had to rely on a scant supply of documents, some held by authorities in the Soviet Union and countries behind the Iron Curtain. The suspects worked hard to hide their identities. Some snuck into the U.S. as refugees, and all denied any role in atrocities. Witnesses were few, particularly in cases like Trawniki, where Nazis wiped out entire populations.

"Our World War II cases are the ultimate cold cases. All of the crimes took place many decades ago on the other side of a vast ocean," Rosenbaum says. "It's very much a search for the proverbial needle in a haystack."

Now, there's also the race against time. People who would have been adults in the Nazi era are quickly dying off.

But Rosenbaum is resolute that pursuing every last Nazi is the least owed to the victims of the Holocaust. In the case of Palij, he says: "What Mr. Palij did prevented other people from reaching old age. He served at the Trawniki SS training and base camp -- really a school for mass murder -- and he trained on live Jews at the adjacent Trawniki Jewish Labor Camp. And, in the end, everyone who was held there was massacred."

The U.S. Holocaust Memorial Museum has helped support the work of Rosenbaum by providing access to documents it had collected on war crimes.

Peter Black, senior historian at the U.S. Holocaust Memorial Museum, worked with Rosenbaum at the Justice Department.

"For Eli, this became a mission," Black says.

That the human rights unit stuck by the work is remarkable because of the many frustrations that arose, Black says. At the height of the Cold War, U.S. courts didn't trust evidence that came from the Soviet Union. U.S. intelligence agencies also warned that some allegations that originated from the Soviets was intended to hurt exile communities from the Baltic countries and Ukraine, which were under Soviet control.

"You can't just decide that because someone gets away with [atrocities] for 40 years, that they are innocent," Black says. "You have to do what you can to achieve justice for the victims, as a statement about our response to these crimes."

Rosenbaum and other lawyers and researchers spent years poring over documents looking to connect suspected Nazis to specific crimes.

The atrocities of the Trawniki camp, where Palij worked, aren't well known in part because the killing was thorough, historians say. One document researchers uncovered helped illustrate the extent of the killing. A soldier broke the butt of his rifle, which meant he was required to file a report so the German SS would issue him a new one. The report mentioned an operation that killed 4,000 people at Trawniki, mostly Jews.

The irony for lawyers and researchers working on the cases was that after stitching together evidence that someone was involved in atrocities, the most U.S. law would allow them to do is to deport former Nazis. Trying them for the actual crimes against humanity is something that was left for authorities in other countries.