Pressure is mounting on the Lake District national park to reconsider its decision to allow off-road cars and motorbikes on farm tracks that campaigners say is devastating the tranquillity of the area.

England’s largest national park has received a highly critical letter from advisers to Unesco, which granted it world heritage status in 2017 after a 20-year campaign.

The problem is particularly acute in Tilberthwaite valley and the valleys surrounding the villages of Little Langdale and Elterwater, and local people have been lobbying for a ban with the backing of the National Trust and a petition signed by more than 350,000 people.

Despite this, in October the Lake District National Park Authority (LDNPA) decided to continue allowing recreational off-road vehicles on dozens of farm tracks on this land – much of it once owned by Beatrix Potter and described by Alfred Wainwright as “the loveliest in Lakeland”.

The authority has received a letter from the International Council on Monuments and Sites (Icomos), the body that advises Unesco on protecting world heritage sites. It says the impact of these vehicles “is not limited to the material aspects of the roads and their erosion but also on the character and tranquillity of the landscape”.

“Icomos states that the traffic regulation orders (TROs), which have been available to national parks to remove the right to use motorised vehicles on certain ‘green’ roads since 2006, appear to be an appropriate tool to deal with the impact,” it adds.

The letter then cites a report published by Icomos on the issue this year. In the report, Icomos found that an increase in the number of vehicles had damaged the “aesthetic and historic qualities” of the landscape. It criticised the national park authority for disregarding complaints and underplaying the damage being inflicted on the area.

It alleged that undue weight had been given to the commercial benefits of allowing tourist trips in 4x4s to remote beauty spots. It also recommended that the park authority should “consider the banning of the use of 4x4 vehicles on green roads within the world heritage property”.

Mechtild Rössler, the director of the Unesco World Heritage Centre, said the authority would not lose its world heritage status but Unesco was monitoring events closely.

She added: “While the Unesco World Heritage Centre follows the situation at the Lake District world heritage site closely with the authorities concerned and our advisory bodies, the world heritage status is not at risk.”

The letter was sent to the Department for Digital, Culture, Media and Sport in September. However, the LDNPA did not receive the document until 8 October, the same day its rights of way committee took the decision to continue allowing recreational vehicles on its lanes.

After the decision, the National Trust issued a strongly worded statement saying it could not support the park authority and urged it to review its decision.

“We think that this activity is at odds with what people want from their national parks in the 21st century – to tackle climate change and champion sustainable transport … We fully understand that managing a national park is challenging but where conflict can’t be resolved by management, protecting the environment is more important.”

The main complaints centre on two tracks near High Tilberthwaite farm, bequeathed to the National Trust by Potter. Campaigners estimate there has been an increase in the number of 4x4s in the area from 90 a month in 2008 to 400 a month in 2017.

Two hill farmers, Glen and Dorothy Wilkinson, left Tilberthwaite farm in April last year because of the damage caused to an unsealed track that runs through the land, saying it had made it “impossible to carry on with our jobs”.

During a meeting at its headquarters in Kendal, the LDNPA voted five to one to back an officers’ report that called instead for a new management plan.

A spokesperson for the authority confirmed the Icomos letter had arrived after the public rights of way committee made its decision.

However, the LDNPA stated the letter did not “raise anything new” and the heritage body’s report had been reviewed fully by the committee as part of its deliberations.