When the children at Barlby primary in west London got up to perform at the Royal Albert Hall last month it was, according to their headteacher, an overwhelming and aspirational event.

The school, which is part of the local community around Grenfell Tower, is among the most diverse and disadvantaged in the country, with more than 25 languages spoken and high levels of pupil premium entitlement.

Its pupils are still talking about the opportunity they had to perform a specially written composition commissioned by three London councils, the historic venue and the Royal College of Music.

Even though their school is in the Royal Borough of Kensington and Chelsea many of the children may not even have known the venue existed, says Anthony Mannix, their headteacher. “They all came back and asked if they could sing in school assembly. It was such a prestigious venue; a huge undertaking and they were in awe of the live instruments. They are still talking about it.”

But, according to research published today, the experience of Barlby’s pupils may be increasingly rare. Authors of The State of Play, a report by the Musicians’ Union and supported by UK Music and the Music Industries Association, describe music education as being in “a perilous state”.

Eight years after ministers published a national plan for music with the aim of ensuring every child had the opportunity to learn a musical instrument, and the establishment of government-funded music hubs – partnerships between schools and arts organisations in their areas – confidence in the government’s handling of music education appears to have collapsed in many places.

A poll of more than 1,000 heads, teachers, music service managers and instrumental teachers suggests that while music education has improved in some areas, there is patchy provision nationwide. Some 97% of classroom music teachers lacked confidence in the government’s handling of it.

The report paints a picture of creeping cuts to music education, a demoralised workforce with poor employment conditions and huge inequality in instrumental provision, with children from families earning under £28,000 a year half as likely to learn a musical instrument as those with a family income above £48,000. And 89% of parents are making a financial contribution towards instrumental lessons.

Curriculum changes and freedoms introduced through widespread academisation, aligned to new accountability measures such as the Ebacc suite of subjects (which do not include music) and an increase in GCSE courses starting in year 9, mean that music is being squeezed out of the curriculum, while the number of postgraduate students choosing to train as music teachers has shrunk by over two-thirds in the past decade.

Of those surveyed, 60% said the Ebacc introduction had directly affected music provision in their schools, which confirms recent findings from the Education Policy Institute that arts entries at GCSE were declining, with a marked north-south and gender divide.

Jonathan Savage, one of the report’s authors and a reader in education at Manchester Metropolitan University, acknowledges that many schools have fantastic music provision. “But for every one that is fantastic, you will find another school in which next to nothing is going on,” he says.

“One of the reasons is that autonomy has taken priority over so much else and headteacher decisions about a subject that is part of the national curriculum are not being challenged robustly enough by the government.”

The report includes examples from practitioners around the country about the pressure they are under from accountability measures, funding and school leaders’ commitment to music education.

Anthony Mannix, headteacher of Barlby primary, with music teacher Simi Khanna. Photograph: Sean Smith/The Guardian

One respondent quoted a letter written by a head to parents, which stated: “Music is a hobby, it is not a career. It will not be supported by the school. I will not allow children to leave school to take graded exams. We are only supporting children’s learning.”

Savage, who is also chair of a music hub in the north-west of England, says the hubs can do exceptional work – in some areas developing a local primary music curriculum, for example – but believes without a refocus on music as an entitlement for all children at school, they may only benefit a minority of children.

Stuart Whatmore helped to organise the Royal Albert Hall event and leads the Tri-Borough Music Hub comprised of the three London boroughs of Westminster, Hammersmith and Fulham and Kensington and Chelsea. He agrees that headteacher buy-in is critical.

“Each year we work with around 90% of our schools in some capacity. That work, advocating and demonstrating how we can support schools, heads and governors in the delivery of their own music curriculum, is at the heart of what we do,” he says. We also offer a wide range of musical learning opportunities that are open to all children who live or go to school in the boroughs, and we intend to continue to evolve to ensure that we are delivering inclusive, life-enhancing and progressive opportunities for all. But schools need to have the vision and understanding. If a school doesn’t want to be involved there is nothing we can do about it.”

Mannix admits that heads are under a lot of competing pressures. “I do have sympathy with heads in other areas. The national context is difficult. Schools have to resist temptations to narrow the curriculum because of the inevitable focus on data and Sats results, which don’t demonstrate the full breadth of what children have enjoyed. But we believe that music has an impact on other subjects, supports learning and builds confidence. We need to think about sending our pupils to secondary with broader knowledge and understanding behind them as well as high academic standards. And we are very well supported by the work of the local music hub in developing staff, sharing knowledge and expertise,” he adds.

The report argues that the government needs to provide an urgent and coherent response to rectify the impact of what it describes as chaotic education policies. It makes more than 30 recommendations, including strengthening music hubs, developing music in teacher training and professional development and, above all, holding schools to account more rigorously for the delivery of music as a universal entitlement during the school day as part of the national curriculum.

Savage says: “The idea of comprehensive music education being replaced by giving pupils the chance to play an instrument for a short time, maybe a term, is not the model of music education that many of us feel is right for a national curriculum subject in school.”

“It needs to be taken much more seriously, be properly resourced with investment in staff and supported by heads, otherwise the experience of each child will depend on where they are educated and the competence of staff. Our recommendations are built around music being a core subject and the best way to reach every child is through the provision of each school,” Savage adds.

The Department for Education is devising a model music curriculum. A spokesperson said: “We want all pupils to have the opportunity to study music – that’s why it is compulsory in the national curriculum from the age of 5 to 14. Analysis from last year shows that through our music hubs programme, more than 700,000 children learnt to play instruments in class together in 2016/17. We are putting more money into arts education programmes than any subject other than PE.”