The managers of Madonna's charity in Malawi have been ousted after they squandered $3.8m (£2.4m) on a school that will never be built, it has been reported.

The damning audit came as Raising Malawi confirmed that it has scrapped plans for a $15m elite academy for girls.

The charity's executive director, Philippe van den Bossche, the partner of Madonna's former personal trainer, left in October after criticism of his management style and spending at the school, according to the New York Times.

"These included what auditors described as outlandish expenditures on salaries, cars, office space and golf course membership, free housing and a car and driver for the school's director," the paper said.

In a shakeup at the charity, the board of directors has been removed and replaced by a caretaker board that includes the 52-year-old singer and her manager, it added.

The abandonment of the Raising Malawi Academy for Girls – backed by prominent Hollywood figures and Madonna's associates in Kabbalah – was announced in January. This caught the Malawian government by surprise and caused anger among villagers who had surrendered their homes to make way for a 117-acre construction site near the capital, Lilongwe.

When the project began to turn sour, Madonna, who has reportedly invested $11m, turned to the Global Philanthropy Group for help. In its report on Raising Malawi's work, seen by the New York Times, it was critical of the way Bossche oversaw both the building plans and the curriculum.

"Philippe's level of mismanagement and lack of oversight was extreme in both aspects of the project and the lack of success of the players on the ground is in large part a result of his inability to effectively manage project plans, people and finances," it was quoted as saying.

The audit also gave a withering verdict on Anjimile Oponyo, who was supposed to head the school. "Her charisma masks a lack of substantive knowledge of the practical application of educational development, and her weak management skills are a major contributor to the current financial and programmatic chaos."

Trevor Neilson, a founder of the group, said $3.8m had been spent on the unbuilt school, much of it on architects, salaries and two cars for employees who had not yet been appointed.

"Despite [this outlay], the project has not broken ground, there was no title to the land and there was, overall, a startling lack of accountability on the part of the management team in Malawi and the management team in the United States," he was quoted as saying.

Construction of the academy was also delayed over a dispute between Raising Malawi and villagers who claimed they were not adequately compensated for land. The building had been due for completion in December this year.

Michael Berg, a co-director of the Kabbalah Centre and co-founder of Raising Malawi, accepted defeat this week. He wrote to the centre's contributors: "A thoughtful decision has been made to discontinue plans for the Raising Malawi Academy for Girls, as it was originally conceived."

But Madonna, who has adopted a boy and a girl from Malawi, insisted the charity will continue to work in the country. "There's a real education crisis in Malawi," she said. "Sixty-seven per cent of girls don't go to secondary school, and this is simply unacceptable. Our team is going to work hard to address this in every way we can."