Ministers are scrapping a requirement for teachers to record instances when they use physical force, as part of a wider move to "restore adult authority" in the wake of the riots in England.

The education secretary, Michael Gove, said that he wanted greater numbers of men teaching, particularly in primary schools, so as to provide children with male authority figures who could display "both strength and sensitivity".

In a speech delivered at Durand academy, in Stockwell, south London, Gove said the regulations on the use of force inhibited teachers' judgment.

He said: "So let me be crystal clear, if any parent now hears a school say, 'sorry, we can't physically touch the students', then that school is wrong. Plain wrong. The rules of the game have changed."

Gove said men considering teaching were deterred by a fear of rules that made contact between adults and children "a legal minefield".

The government was planning to start a programme this autumn encouraging former members of the armed forces to take up teaching, specifically to ensure more male role models, Gove said.

In a speech that sought to address the causes of the riots in August, Gove began by making a moral distinction between what he called a "hard-working majority" and a "vicious, lawless, immoral minority". But he went on to examine what he said were the policy failures that lay behind the creation of the "educational underclass".

He said: "To investigate where the looters came from is not to make excuses because of background. It is to shine a light on failures that originated in poor policy, skewed priorities, and the deliberate undermining of legitimate authority."

Gove said he was haunted by the thought that, if circumstances had been different, he might have been a part of this underclass. The education secretary highlighted his own family background. "I was born to a single parent, never knew my biological father and spent my first few months in care.

"Thanks to the love of my adoptive mother and father, and the education I enjoyed, I was given amazing opportunities. So I know just how much the right parenting, the right values at home, and the right sort of school matter in determining a child's fate."

Gove said there had been a slow erosion of adult authority, subverted by a culture in which young people felt able to ignore civilised boundaries. "The only way to reverse this dissolution of legitimate authority is step-by-step to move the ratchet back in favour of teachers."

Gove also spoke of an "iron-clad link" between illiteracy, disruption, truancy, exclusion and crime.

More than 430,000 children were absent for 15% of school time, and more than a million pupils missed 10% of the academic year, he said.

He added that only a third of those students who missed between 10% and 20% of school got the "basic minimum" of five good GCSE passes.

The government is asking Charlie Taylor, a headteacher and Gove's adviser on behaviour, to look at improving "alternative provision" units for children with behavioural problems.

Taylor will be asked to work with Lord Harris of Peckham, who sponsors academies, to speed up the ability of those entities to create provision for excluded and disruptive pupils.

Brian Lightman, general secretary of the Association of School and College Leaders, welcomed Gove's statement concerning the use of force against pupils.

He said: "ASCL is delighted that the secretary of state has responded to our advice with the wise decision not to proceed with these regulations. The requirement would have imposed yet another bureaucratic burden that did nothing to improve discipline or safeguard children.

"The use of physical restraint is thankfully required very rarely in schools. On occasions where it is needed, detailed guidance exists and staff fully understand the need to follow it to the letter. Schools already keep records of breaches of discipline."