Two-time Oscar-winning actor Michael Caine has weighed in on the Oscars diversity controversy, saying in a recent interview that while he personally believes that one black actor in particular should have been nominated at this year’s ceremony, “you can’t vote for an actor because he’s black.”

The Academy of Motion Picture Arts and Sciences has come under fire for nominating exclusively white actors in top acting categories at this year’s Oscars, marking a repeat of last year’s #OscarsSoWhite controversy. Filmmakers Spike Lee and Michael Moore and actors Will and Jada Pinkett Smith have vowed not to attend the February 28 ceremony in protest, while others have simply spoken out to denounce the Academy for the lack of diversity in its nominating process.

But in an interview with BBC Radio 4’s Nick Robinson, Caine said that the quality of a performance, rather than the color of an actors’ skin, is paramount when considering Oscar nominations.

“There’s loads of black actors,” the 82-year-old Youth star said. “You can’t vote for an actor because he’s black. You can’t just say, ‘I’m going to vote for him. He’s not very good, but he’s black. I’ll vote for him.’ You have to give a good performance.”

Caine, who has been nominated six times for Oscars and won twice, said that he thought Idris Elba would be nominated for his role as the sadistic Commandant in the Netflix war drama Beasts of No Nation. When Robinson pointed out that Elba wasn’t nominated this year, Caine demurred.

“Well, look at me,” Caine said. “I won the [European Film Award] for best actor, and I got nominated for nothing else.” Caine’s film Youth was nominated for just one Oscar, for Best Original Song.

Caine said the best advice he could give to minority actors is to “be patient.”

“Of course it will come,” he said. “It took me years to get an Oscar.”

Since the Academy announced its nominations last week, a slew of actors, actresses and producers have spoken out to blast the awards show for its lack of diversity, including David Oyelowo, George Clooney, Lupita Nyong’o, Dustin Hoffman and Mark Ruffalo. Straight Outta Compton executive producer Will Packer has called the lack of nominations for people of color a “complete embarrassment,” while the Rev. Al Sharpton has called for Americans nationwide to “tune out” of the broadcast in protest.

However, some have defended the Academy in the wake of the media firestorm.

In an interview with Variety, Boyz ‘N The Hood director John Singleton said that “there are only so many slots” for nominations, and those nominations will go to those films and performances that the Academy feels should be recognized. Singleton, who became the first African-American to earn a Best Director nomination for the 1991 film, said that the Academy’s nominating process is “almost like the lottery.”

“It’s like every year people complain,” the director said. “People even complain even when we have a lot of nominations. It is what is is. I’ve been in the game for 25 years. You never know – it’s the luck of the draw for you. To me, I’m not surprised. I’m not disappointed either, as much as other people are disappointed. There’s a whole elevation of work that happens.”

Speaking to the Hollywood Reporter on Friday, Oscar-winning Schindler’s List producer Gerald Molen also defended the Academy and called the growing boycott movement against the Oscars “ridiculous.” Molen said it would be tough to believe that Academy members are not voting for minority actors because of their skin color.

“In a liberal town like Hollywood, that makes about as much sense as saying all members of the Academy vote Republican,” he joked.