The Metropolitan police increased its use of stop and search last year, with a 19% rise among London’s minority black population, which was targeted more than the white population, official figures show.

Analysis commissioned by the Guardian also shows that searches of black people were less likely to detect crime than those conducted on white people, and most stops found no wrongdoing.

Black people make up 15.6% of London’s population while white people make up 59.8%. In 2018, 43% of searches were of black people, while 35.5% were of white people , according to official figures from the London Mayor’s Office for Policing and Crime (MOPAC).

A briefing note for senior officials, seen by the Guardian, says “disproportionality has increased”, with the likelihood of black people being stopped 4.3 times higher than white people in 2018, compared with 2.6 times more likely in 2014, according to figures from MOPAC, which oversees the Met.

The Met defended its actions saying youths from an “African-Caribbean heritage” were more likely than whites to be knife attackers or victims, and it was a crucial way to catch criminals and keep youngsters safe. It said violence “disproportionately affects areas with a more ethnically diverse population”.

A black police leader accused the Met of racial profiling and the MP David Lammy, who was commissioned by the prime minister to investigate racial bias in the criminal justice system, warned police they were unfairly targeting young black men.

Stop and search is controversial because of a historic greater likelihood of use against black communities with a resultant damage in trust and confidence in policing. Theresa May, when home secretary, warned police about the damage such disproportionality caused and pressed for change, resulting in a drop in the overall level.

A total of 151,102 searches were carried out in 2018, an increase of 16% in London on the previous year, rising by 5% for white people and 19% for black people.

Stop and search has been linked by academics and some in policing to outbreaks of disorder, while others have linked the drop to rising crime.

The Metropolitan police has been under pressure to increase the use of stop and search as violent crime rose.

The figures for 2018 shows there were 10.8 stops for every 1,000 white people, while for black people in London it was 50.2 stops per thousand of population.

In a statement police said: “The Met has seen an increase in the use of the tactic over the last year particularly within the last six months, largely due to the increase in street violence and related drug crime.

“Crime is not proportionate and the root causes are complex. . Knife crime and street violence disproportionately affects boys and young men, particularly of African-Caribbean heritage both in terms of victims and perpetrators.

“In 2018, 76% of homicide victims are male, with 62% of the total being of African-Caribbean heritage and aged under 25. In relation to victims of knife injuries under the age of 25, 455 were white and 1,370 were BAME [black and minority ethnic].

“We must focus our efforts … in the areas requiring our support and intervention to stop more young people being injured, dying or damaging their lives by committing serious offences.”

Officers require reasonable grounds to suspect in order to use the power, and the most common reason they gave in 2018 was drugs, in 57% of stops, up 10% compared with the previous year. Stopping someone to search for weapons was the stated reason in 19% of cases, with such stops increasing 36% year on year.

Lammy, who is the Labour MP for Tottenham, said: “Stop and search disproportionately and unfairly targets young black men. It destroys trust between police and the communities they serve.

“Stop and search is consistently ineffective at reducing violent crime and reliance on it gets in the way of long-term solutions to address its root problems.”

Young people in London were the most likely to be searched, with those aged 15-19 years old stopped at a rate of nearly 97 per thousand of population.

Overall 16% of searches led to an arrest and the data shows arrest rates were lower for the black population.

Where an officer stated the reason for the stop was a suspicion a person was carrying weapons, 21% of stops of white people resulted in an arrest. For black people that figure was 16%, the data from MOPAC said.

Overall “positive outcomes” where an offence is detected after a stop, were higher for white people than black. These led to sanctions such as cautions or community resolution, as well as arrest.

For white people, 30.5% of searches resulted in further action, for Asian people 27.8%, and for black people lower still, at 26.7%, according to Dr Krisztián Pósch, from the London School of Economics, who analysed the figures for the Guardian. He said there was a racial effectiveness gap in the searches carried out in 2018 by the Met: “The data shows that police are not just stopping black people more disproportionately, but are less likely to detect crime when they do compared to when they stop white people.”

The Met said there was very little difference in outcomes for different ethnic groups: “We are convinced that stop and search is an effective tactic in preventing and detecting crime. … Over 4,200 weapons were also taken off the streets last year as a direct result of stop and searches.”

Met police commissioner, Cressida Dick, thinks the increase in violence has been halted through an array of tougher tactics, but says longer term solutions are also needed.

Sgt Tola Munro, the president of the National Black Police Association, said: “It is no excuse to suggest that weapons or drugs are more likely to be found on black and Asian people. Even if there were that does not excuse this disproportionality – it’s not unconscious, it’s systemic racial profiling.”

Katrina Ffrench of StopWatch, which campaigns for fairer use of the power, said it was time for the government to legislate, a radical move May threatened the police with when she was home secretary. Ffrench said: “Police forces that persistently discriminate against black and ethnic groups should face sanctions and officers found to be misusing or abusing their stop and search powers should not be permitted to use them.”

The Met said: “Stop and search is also an effective preventative measure. In utilising the tactic consistently in an evidenced based manner in relation to specific crime types, we believe this can also significantly deter criminality.”