David Williams did not have an easy life. He moved to Newburgh, a gritty, impoverished town on the banks of the Hudson an hour or so north of New York, at just 10 years old. For a young, black American boy with a father in jail, trouble was everywhere.

Williams also made bad choices. He ended up going to jail for dealing drugs. When he came out in 2007 he tried to go straight, but money was tight and his brother, Lord, needed cash for a liver transplant. Life is hard in Newburgh if you are poor, have a drug rap and need cash quickly.

His aunt, Alicia McWilliams, was honest about the tough streets her nephew was dealing with. "Newburgh is a hard place," she said. So it was perhaps no surprise that in May, 2009, David Williams was arrested again and hit with a 25-year jail sentence. But it was not for drugs offences. Or any other common crime. Instead Williams and three other struggling local men beset by drug, criminal and mental health issues were convicted of an Islamic terrorist plot to blow up Jewish synagogues and shoot down military jets with missiles.

Even more shocking was that the organisation, money, weapons and motivation for this plot did not come from real Islamic terrorists. It came from the FBI, and an informant paid to pose as a terrorist mastermind paying big bucks for help in carrying out an attack. For McWilliams, her own government had actually cajoled and paid her beloved nephew into being a terrorist, created a fake plot and then jailed him for it. "I feel like I am in the Twilight Zone," she told the Guardian.

Lawyers for the so-called Newburgh Four have now launched an appeal that will be held early next year. Advocates hope the case offers the best chance of exposing the issue of FBI "entrapment" in terror cases. "We have as close to a legal entrapment case as I have ever seen," said Susanne Brody, who represents another Newburgh defendant, Onta Williams.

Some experts agree. "The target, the motive, the ideology and the plot were all led by the FBI," said Karen Greenberg, a law professor at Fordham University in New York, who specialises in studying the new FBI tactics.

But the issue is one that stretches far beyond Newburgh. Critics say the FBI is running a sting operation across America, targeting – to a large extent – the Muslim community by luring people into fake terror plots. FBI bureaux send informants to trawl through Muslim communities, hang out in mosques and community centres, and talk of radical Islam in order to identify possible targets sympathetic to such ideals. Or they will respond to the most bizarre of tip-offs, including, in one case, a man who claimed to have seen terror chief Ayman al-Zawahiri living in northern California in the late 1990s.

That tipster was quickly hired as a well-paid informant. If suitable suspects are identified, FBI agents then run a sting, often creating a fake terror plot in which it helps supply weapons and targets. Then, dramatic arrests are made, press conferences held and lengthy convictions secured.

But what is not clear is if many real, actual terrorists are involved.

The homes of the Fort Dix Five were raided by the FBI. Photograph: Joseph Kaczmarek/AP

Another "entrapment" case is on the radar too. The Fort Dix Five – accused of plotting to attack a New Jersey army base – have also appealed against their convictions. That case too involved dubious use of paid informants, an apparent over-reach of evidence and a plot that seemed suggested by the government.

Burim Duka, whose three brothers were jailed for life for their part in the scheme, insists they did not know they were part of a terror plot and were just buying guns for shooting holidays in a deal arranged by a friend. The "friend" was an informant who had persuaded another man of a desire to attack Fort Dix.

Duka is convinced his brothers' appeal has a good chance. "I am hopeful," he told the Guardian.

But things may not be that easy. At issue is the word "entrapment", which has two definitions. There is the common usage, where a citizen might see FBI operations as deliberate traps manipulating unwary people who otherwise were unlikely to become terrorists. Then there is the legal definition of entrapment, where the prosecution merely has to show a subject was predisposed to carry out the actions they later are accused of.

Theoretically, a simple expression, like support for jihad, might suffice, and in post-9/11 America neither judges nor juries tend to be nuanced in terror trials. "Legally, you have to use the word entrapment very carefully. It is a very strict legal term," said Greenberg.

But in its commonly understood usage, FBI entrapment is a widespread tactic. Within days of the 9/11 terror attacks, FBI director Robert Mueller issued a memo on a new policy of "forward leaning – preventative – prosecutions".

Central to that is a growing informant network. The FBI is not choosy about the people it uses. Some have criminal records, including attempted murder or drug dealing or fraud. They are often paid six-figure sums, which critics say creates a motivation to entrap targets. Some are motivated by the promise of debts forgiven or immigration violations wiped clean. There has also been a relaxing of rules on what criteria the FBI needs to launch an investigation.

Often they just seem to be "fishing expeditions". In the Newburgh case, the men involved met FBI informant Shahed Hussain simply because he happened to infiltrate their mosque. In southern California, FBI informant Craig Monteilh trawled mosques posing as a Muslim and tried to act as a magnet for potential radicals.

Monteilh, who bugged scores of people, is a convicted felon with serious drug charges to his name. His operation turned up nothing. But Monteilh's professed terrorist sympathy so unnerved his Muslim targets that they got a restraining order against him and alerted the FBI, not realising Monteilh was actually working on the bureau's behalf.

Muslim civil rights groups have warned of a feeling of being hounded and threatened by the FBI, triggering a natural fear of the authorities among people that should be a vital defence against real terror attacks. But FBI tactics could now be putting off many people from reporting tip-offs or suspicious individuals.

"They are making mosques suspicious of anybody. They are putting fear into these communities," said Greenberg. Civil liberties groups are also concerned, seeing some FBI tactics as using terrorism to justify more power. "We are still seeing an expansion of these tools. It is a terrible prospect," said Mike German, an expert at the American Civil Liberties Union and a former FBI agent who has worked in counter-terrorism.

German said suspects convicted of plotting terror attacks in some recent FBI cases bore little resemblance to the profile of most terrorist cells. "Most of these suspect terrorists had no access to weapons unless the government provided them. I would say that showed they were not the biggest threat to the US," German said.

"Most terrorists have links to foreign terrorist groups and have trained in terrorism training camps. Perhaps FBI resources should be spent finding those guys."

Also, some of the most serious terrorist attacks carried out in the US since 9/11 have revolved around "lone wolf" actions, not the sort of conspiracy plots the FBI have been striving to combat. The 2010 Times Square bomber, Faisal Shahzad, only came to light after his car bomb failed to go off properly. The Fort Hood killer Nidal Malik Hasan, who shot dead 13 people on a Texas army base in 2009, was only discovered after he started firing. Both evaded the radar of an FBI expending resources setting up fictional crimes and then prosecuting those involved.

Yet, as advocates for those caught up in "entrapment" cases discover, there is little public or judicial sympathy for them. Even in cases where judges have admitted FBI tactics have raised serious questions, there has been no hesitation in returning guilty verdicts, handing down lengthy sentences and dismissing appeals.

The Liberty City Seven are a case in point. The 2006 case involved an informant, Elie Assaad, with a dubious past (he was once arrested, but not charged, for beating his pregnant wife). Assaad was let loose with another informant on a group of men in Liberty City, a poor, predominantly black, suburb of Miami. The targets were followers of a cult-like group called The Seas of David, led by former Guardian Angel Narseal Batiste.

The group was, perhaps, not even Muslim, as its religious practices involved Bible study and wearing the Star of David. Yet Assaad posed as an Al-Qaida operative, and got members of the group to swear allegiance. Transcripts of the "oath-taking" ceremony are almost farcical. Batiste repeatedly queries the idea and appears bullied into it. In effect, defence lawyers argued, the men were confused, impoverished members of an obscure cult.

Yet targets the group supposedly entertained attacking included the Sears Tower in Chicago, Hollywood movie studios and the Empire State Building. Even zealous prosecutors, painting a picture of dedicated Islamic terrorists, admitted any potential plots were "aspirational", given the group had no means to carry them out.

Nonetheless, they were charged with seeking to wage war against America, plotting to destroy buildings and supporting terrorism. Five of them got long jail sentences. Assaad, who was recently arrested in Texas for attempting to run over a policeman, was paid $85,000 for his work.

This year the jailed Liberty City men launched an appeal and last week judgment was handed down. They lost, and officially remain Islamic terrorists hell-bent on destroying America. Not that their supporters see it that way.

"Our country is no safer as a result of the prosecution of these seven impoverished young men from Liberty City," said Batiste's lawyer, Ana Jhones.

"This prosecution came at great financial cost to our government, and at a terrible emotional cost to these defendants and their families. It is my sincere belief that our country is less safe as a result of the government's actions in this case."