On 17 January, a masked assailant threw sulphuric acid into the face of the artistic director of the Bolshoi Ballet, Sergei Filin – an attack so violent it made news around the world. The case continues to grip, surrounded by rumours and counter-rumours. Last week, Bolshoi dancer Pavel Dmitrichenko apparently confessed to instigating the attack, via an accomplice; both were charged. This week, his Bolshoi colleagues have written an open letter to Vladimir Putin arguing that Dmitrichenko was pressured into the confession, and demanding an independent investigation.

Why was Filin targeted so savagely? Some have pointed to the Bolshoi's long history of artistic conflicts – some dancers and choreographers eager to embrace modern influences, others loyal to the old Soviet-era Bolshoi. Filin was a reforming figure; before him, director Alexei Ratmansky and his potential successor Gennady Yanin were subject to campaigns of threat and vilification. In 2011, nearly 200 explicit photographs of Yanin were emailed anonymously to colleagues and press around the world.

There has been other speculation, including rumours of financial corruption. In court, Dmitrichenko accused Filin of favouritism when it came to allotting grants. The director's critics say he has alienated his company through cavalier casting decisions; there was controversy over the 2011 hiring of the American dancer David Hallberg.

Yet even the most colourful allegations seem bizarrely disproportionate to the enormity of the assault. Few seem to believe Dmitrichenko could have been acting alone. Members of the Bolshoi's management, and several investigators, argue that even if the dancer was involved, he was being manipulated by other ambitious figures within the Bolshoi – people willing to go to any lengths to seize control of the company themselves.

Ballet really matters in Russia. "That's very good for us," says Bolshoi spokeswoman Katerina Novikova. "But it can also be part of our problem." For generations, tsars spent lavishly on the two main companies, the Mariinsky in St Petersburg and the Bolshoi in Moscow. During the Soviet era, Stalin made regular use of his private box at the Bolshoi, promoting it as a "people's art". There are close personal links between the Kremlin and a select band of dancers and choreographers.

Ballet is also a source of passion and pride among ordinary Russians. According to Xander Parish, the young British dancer who made history by joining the Mariinsky three years ago, "there are posters for ballet performances all around St Petersburg, and ballet documentaries are always being shown on TV. In the theatre, I see people who look as though they don't know where their next meal is coming from. But there's an expression on their faces, an intensity that I don't think I ever saw at Covent Garden."

The Spanish choreographer Nacho Duato, who now lives in St Petersburg, concurs. "You feel a reverence here that's very special. When you walk into the Vaganova school [the city's 250-year-old ballet academy], it's as though you're entering a temple."

There's also a new, or newly renovated, ballet company on the block. In 2007, Russian billionaire Vladimir Kekhman, who made his fortune from fruit importation (his nickname is the Banana King), was casting around for a project to absorb his profits: instead of a football club or a newspaper, he chose a ballet company.

The Mikhailovsky theatre in St Petersburg was founded in 1833, as a stage for all genres of live performance. Although in the shadow of its grander rival, its grander rival, the Mariinsky, it came to prominence during the early 20th century as a home for experimental ballet and opera, producing works such as Shostakovich's The Bright Stream. By the time Kekhman set his sights on it, it was languishing artistically, making it perfect for reinvention.

Appointing himself general manager, Kekhman spent nearly $60m on restoring the building and acquiring star names. He poached the golden Bolshoi couple Natalia Osipova and Ivan Vasiliev; he appointed Duato as artistic director and creator of a new contemporary repertory; and he employed the renowned ballet master Mikhail Messerer to take care of the 19th- and 20th-century classics.

In an impressively short period, Kerkham confounded the sceptics who assumed he'd be running the company like a football team. At home, they now sell out every performance; this month, they return to London for their third visit in five years, with a programme of seven ballets and an exceptional roster of dancers.

It's a remarkable success story, and when I visited the company in St Petersburg last week, it became clear how much of that is down to Kekhman's money. The drama at the Bolshoi is symptomatic of a much larger turbulence within Russian ballet, as it makes the difficult transition from the Soviet era to the 21st century. The Mikhailovsky has the freedom to leapfrog over the problems with which the larger, state-funded companies still struggle.

The key issue is the collision between the forces of conservatism and change. At the Bolshoi, a core faction remains passionately loyal to Yuri Grigorovich, the autocratic director who, between 1964 and 1995, imposed his own broad, athletic style of choreography on the company. When subsequent directors, such as Ratmansky, have attempted to introduce a more varied repertory, those opposed have invoked Grigorovich's legacy as their rallying cry.

One of the most vocal refuseniks has been Nikolai Tsiskaridze, a flamboyant dancer with a huge media profile who has made no secret of his own ambition to run the Bolshoi. Even though Filin has proved accommodating to Grigorovich, allowing the 86-year-old choreographer to oversee revivals of his work, Tsiskaridze has nonetheless worked the old pro- and anti-Grigorovich loyalties to whip up opposition.

At the Mariinsky, the divisions are different, with conservatives focusing on the famous academic purity of the company's style. If the Bolshoi's trademark is Grigovorich's blunt, swaggering Spartacus, the Mariinsky is known for its immaculate productions of 19th-century classics such as Sleeping Beauty and Swan Lake.

These don't exclude the acquisition of newer ballets, including those of postmodernist choreographer William Forsythe. According to Parish, when auditions were held for the casting of Forsythe's male quartet NNNN, "there was standing room only in the studio".

Yet these acquisitions were too much of a rarity to satisfy the former Mariinsky principal Leonid Sarafanov, who defected to the Mikhailovsky in 2011. "No company dances the classics as well as the Mariinsky, but for me it felt like a dead end. What was left for me to do? Some extra pirouettes in my Swan Lake variation? To work closely with a choreographer like Nacho is like hitting the jackpot. With him it's a completely different way of using your body, a different plastique. I can feel myself dancing much better now."

The Mikhailovsky has also sidestepped some of the more inflammatory issues surrounding dancers' working conditions. In the last two decades, principals in the two big companies have observed the higher levels of pay enjoyed by western dancers and gone hunting after foreign appearances. To counter this trend, directors have instituted a system whereby extra fees are paid for every performance these dancers give at home. It's a system that has exacerbated internal rivalries: at the Bolshoi, with more than 200 dancers in its ranks, competition for leading roles – and the extra money – is intense.

At the Mikhailovsky, which is half the size, there is less competition. Sarafanov says he dances more in one month than he used to in half a year. At the same time, the workload is more evenly distributed through the company. At the Mariinsky, it is not uncommon for junior dancers to work for four to six weeks without a day off – a situation that recently drove them to sign a petition of complaint.

It would be wrong to suggest that Kekhman's money has entirely liberated his company from trouble. Last year, he and his business were declared bankrupt, and he is under police investigation for fraud. Many argue that the Banana King's passion for ballet began when he felt the need to put a benign public face to his business practices.

Yet when I talk to insiders at the Mikhailovsky, it's hard to discern anything but optimism. Kekhman has been careful to ensure that the theatre is now independent of his money – orchestrating a series of private sponsors who supplement the company's modest state funding. One potential glitch in his vision is the fact that Duato has been invited to direct Berlin State Ballet from 2014. He plans to continue as resident choreographer at the Mikhailovsky; there is no hint as to who might replace him as director. As ballet master, Messerer remains a stabilising force, with plans to extend the repertory to include works from the Royal Ballet by Frederick Ashton, Kenneth MacMillan and some of the younger choreographers.

Certainly there is no sign of the Mikhailovsky's imported stars regretting their decision to join. Meanwhile, over at the Bolshoi, spokeswoman Novikova is observing the company's development with interest. "They're really reinventing themselves," she says. "They're lucky not to be tied down by tradition like us." Nor by a court case for grievous bodily harm: on Tuesday, Filin told Russian state TV he believed Dmitrichenko was part of a "narrow circle of people who openly disliked me and threatened me and many other people". He has 50% vision in one eye, less in the other. A date for the dancer's trial has not yet been set, and he remains on the Bolshoi payroll.

• Judith Mackrell travelled to St Petersburg as a guest of the Mikhailovsky Ballet