Schools are attempting to manipulate the exams system by issuing spurious appeals against results in an effort to improve their pupils' grades, the qualifications regulator has said.

Ofqual suggested that the GCSE and A-level appeals process was designed for a "more innocent era" and was being used tactically by teachers under pressure to secure good results.

Its report appears to back claims from headteachers that staff are under excessive stress from the Department for Education to improve pupils' exam grades.

Ofqual said evidence suggested an increase in appeals against results. Examiners dealing with appeals may be looking for extra marks to award to students as they are conscious that the final result could have a major impact on a youngster's future, it found.

Glenys Stacey, the chief regulator, said the appeals system was "not good enough" and Ofqual would launch a "fundamental redesign" of the process.

"The analysis … has shown that there is tactical appealing at critical grade boundaries – C/D at GCSE and B/A at A-level. That doesn't mean that every appeal at that boundary is tactical, but you can see from the pattern that it would suggest there is tactical appealing.

"Secondly, the way the appeal system is designed at the moment, and indeed headteachers agree with us, is that where you're within a couple of marks of such a grade boundary it's worth appealing because it is a one-way bet. As a regulator we need to look at that and see if we can design an appeals system which is fair, transparent and effective," she said.

Figures show that 2.3% of exam papers were sent back for appeal last year, compared with 1% in 2009 and 1.9% in 2012. Ofqual's report says the appeals system does not have the confidence of schools and colleges, and is "both overly complex and conceptually at odds with how qualifications are marked initially".

It adds: "Moreover, we found schools and colleges were using it tactically due to pressures to deliver results, particularly at the top grades at A-level and the C/D grade boundary at GCSE. The system was not designed to be used in this way and, while it is coping now, there is a risk it could bend under the pressure in time."

The report says some teachers have admitted using the system speculatively if their pupils' results fall just below an important grade boundary. State schools were more likely to challenge marks that were just below a C grade in key subjects such as English and maths, while private schools appealed against results at the A-level A/B and A*/A thresholds in an attempt to secure university places.

The regulator found anecdotal evidence that exam boards believed that examiners may look for extra marks to award students when dealing with an appeal, despite being briefed not to do so.

Brian Lightman, general secretary of the Association of School and College Leaders, said: "We are disappointed that the report gives no indication of the importance of engaging with school leaders on proposed changes. Changes must be fair to all candidates and the right to appeal against an initial decision must be retained."

William Richardson, general secretary of the Headmasters' and Headmistresses' Conference, said: "We are pleased that after 17 months, Ofqual has acknowledged and begun to address some of the failings that HMC has been highlighting.

"For too long, supervision of the examinations industry in England has failed teachers, parents and, most importantly, students. This has been due to permissive attitudes toward the exam boards – one that is insufficient to ensure consistency and quality in marking and grading of exams or in the fairness and rigour of the appeals process."