LOS ANGELES -- An Egyptian man was held without bail Tuesday on federal charges of fraudulently collecting insurance payouts after driving his severely autistic children to their deaths when he sped off a pier into the Port of Los Angeles.

Ali Elmezayen plotted for over two years before launching his plan into action April 9, 2015, when he barreled down a commercial fishing dock in San Pedro, turned his Honda abruptly toward a parking space and gunned the engine, federal authorities said.

"Witnesses heard the tires of the car screech loudly and saw the car accelerate before it flew off the edge of the wharf, crashed into the water and immediately sunk," FBI agent Matthew Parker wrote in an affidavit in support of Elmezayen's arrest last week.

Get Breaking News Delivered to Your Inbox

Elmezayen's car window was open, and he surfaced within 30 seconds and swam to a ladder on the pier. His partner, Rabab Diab, couldn't swim, but she was rescued by a fisherman after surfacing and screaming, "My kids, my kids."

Two boys, Abdelkarim, 8, and Elhassan, 13, couldn't swim and were strapped in child seats. They were pulled lifeless from the car by rescue divers and died from drowning.

Elmezayen collected more than $260,000 in insurance payouts for the deaths and wired more than $170,000 back to his native Egypt, prosecutors said.

Elmezayen, 44, did not enter a plea in the case in U.S. District Court, but he has told investigators and others he may have accidentally accelerated instead of hitting the brake, according to court papers. He blamed the city, Honda, and Jiffy Lube and others for negligence.

"In short, there were functionally no precautionary measures taken by any public defendant to this action to prevent a car from plunging into the deep waters below the berth," a lawsuit filed in 2016 stated, CBS Los Angeles reported. "This (also) included a lack of adequate signage or other such warnings to the public."

His lawsuit was tossed out.

The Los Angeles County district attorney declined to bring charges in the deaths because of insufficient evidence. A police mechanic found evidence that the brake pedal didn't properly work, but couldn't tell if that was the case before the tragedy or because of salt water corrosion.

While a county prosecutor said the father's explanation could create reasonable doubt, they suggested police tell federal prosecutors about evidence of insurance and other fraud. Federal investigators said they found that Elmezayen spent about a quarter of his reported $24,000 salary to insure himself, his longtime partner and three sons for $6 million.

Elmezayen repeatedly called insurers to confirm they wouldn't automatically investigate a claim made more than two years after a policy was written, federal prosecutors said. The deadly plunge occurred two years and 12 days after the final policy took effect, authorities said.

Prosecutors argued that Elmezayen should not be free awaiting trial because he was likely to flee to his homeland. They also argued he still posed a danger to Diab because he had intended to kill her and still pays premiums on her life insurance policy and that of another son, who was at camp when his siblings were killed.

A public defender representing Elmezayen argued that nothing had happened to Diab and the other son in over three years. He asked to have Elmezayen use $250,000 in equity in two houses he owns to secure his bond.

Elmezayen and Diab came to the U.S. in 2000 and overstayed their tourist visas, prosecutors said. Both married other partners in hopes of becoming permanent residents. Elmezayen has acknowledged his was a sham marriage.

Judge Alka Sagar said Elmezayen lacked credibility and she denied bail, in part, because of what she called the "ghastly scheme to obtain insurance benefits."