Now Tesla has another battle on its hands, this time with unhappy shareholders.

They argue that Tesla's board has failed to hold its charismatic but controversial CEO Elon Musk to account about the company's finances and performance, rubber stamping Musk's decisions instead.

Three Tesla directors are up for a vote at next week's annual meeting, and one activist shareholder firm wants to dump all of them. Among them are Kimbal Musk, brother of CEO Elon Musk. That effort is being led by CtW Investment Group, which works with union-sponsored pension funds.

At the same time, the advisory firm Institutional Shareholder Services is calling to defeat the other two directors who aren't named Musk -- James Murdoch, the CEO of 21st Century Fox (FOX) and son of Rupert Mudoch, and Antonio Gracias, head of Valor Management, an early investor in Tesla.

Related: Here's what you get when you buy Tesla's $35,0000 Model 3

In a letter CtW filed with the SEC early this month, the company wrote, "We admire Tesla as a successful innovator with an environmental mission; however, its continuing success looks more tenuous than ever. Especially in light of CEO Elon Musk's startling refusal to answer straightforward questions from analysts on a recent investor call, Tesla shareholders need to let the board know that it must raise its game."

Tesla also faces a shareholder resolution calling for the automaker to split jobs of chairman and CEO between different people, rather than allowing Elon Musk to hold both positions.

The shareholder's meeting is set for June 5.

Tesla has been under fire for much of this year.

The automaker has struggled to ramp up production of its new of Model 3 sedans, and that's stoked questions as to whether Tesla could run out of cash later this year. The company's credit rating has been reduced further into junk bond status and its once high-flying stock is down 7% for the year.

Still, Musk has dismissed all of these concerns repeatedly, and made a bold promise to turn a profit in both the third and fourth quarter of this year.

Efforts like CtW's campaign virtually always fall short. But Tesla is taking the matter seriously, responding Tuesday evening in its own letter to shareholders defending the board members up for election.

Related: Elon Musk has more to say about the media

In its letter, CtW argues that Kimbal Musk's "renomination goes to the heart of the problems with Tesla's board: Too many of these directors, including all three of this years' nominees, are incapable or unwilling to contradict Elon Musk's whims."

CtW also objects to Gracias being considered an independent director, given his long-standing business relations with Musk. The letter said Gracias has invested in a number of Musk ventures, including SpaceX, the privately held rocket company that also has Musk as CEO. Gracias is also a director of that company.

Finally, CtW says that Murdoch doesn't have the needed expertise in manufacturing and engineering and has worked almost exclusively for companies in which his father is the controlling shareholder. It points out that Tesla recently added another media executive, Linda Johnson Rice of CEO of Ebony Media and Johnson Publishing, to its board.

"It is not clear why Tesla would need two directors with media but not automotive backgrounds," said CtW.

But Tesla is defending all three directors, noting that Murdoch is an experienced CEO who has scaled and run large successful global companies. It also says that Gracias is an experienced operator who has run factories and an auto parts company and is advising on ramping up Model 3 production.

And the automaker argues that Kimbal Musk has been a successful entrepreneur and technology investor.