NEW YORK — Yankees center fielder Brett Gardner flipped out when he was ejected after the umpire mistook his voice for Cameron Maybin cursing at him in Toronto on Friday.

This isn’t going to make Gardner any happier.

MLB Chief Baseball Officer Joe Torre, the legendary ex-Yankees manager, said Gardner did, in fact, deserve for plate umpire Chris Segal to toss him.

“But watching enough video and stuff, Gardner earned being thrown out,” Torre said in the dugout at Yankee Stadium on Tuesday afternoon, before the Yankees faced the Orioles. “Maybe not that particular time but he certainly had the right to be thrown out, even though the umpire, I think there was so much carping from the dugout that he pulled the trigger on what he assumed was Gardner.”

Torre was at the stadium on behalf of the Joe Torre Safe At Home Foundation, which raises domestic violence awareness.

On Friday, TV microphones picked up Segal telling manager Aaron Boone that he threw out Gardner after he heard Gardner call him “(bleeping) terrible.”

But replay showed the it was Maybin who yelled that at Segal, not Gardner.

Torre said that considering the things Gardner had said and done before Maybin yelled at Segal, it wasn’t completely unreasonable for Segal believe it was Gardner’s voice instead of Maybin’s. Gardner had also banged his bat repeatedly against the dugout in protest of Segal.

“Yeah, well, that’s all the noise he heard, was from Brett,” Torre said. “Which was a little more surprising for me, because that’s not his personality.”

After Gardner got thrown out, Boone had to restrain him from getting in Segal’s face as he repeatedly yelled that he didn’t say anything. After the game, Gardner was fuming about the incident.

“Maybe 30 seconds goes by and somebody tells me I’ve been tossed," Gardner said at the time. “At that point I went out there to talk to (Segal), tell him I didn’t say anything and he got it wrong. Then he looked at me and told me he saw me say it, or say something, which made me mad, because he’s lying. I didn’t say anything. It’s all on video.”

Torre said he talked with Segal after the incident but didn’t share details of the discussion.

“I know there’s been talk about umpires when they make a mistake or they miss a call they don’t have any repercussions — that’s not true," Torre said. “They watch their game every single day. They get rated, whether it’s on the bases or balls and strikes. And next month, when we meet for postseason, that plays a role, that plays a part, deciding who goes to the postseason.”

Brendan Kuty may be reached at bkuty@njadvancemedia.com. Follow him on Twitter @BrendanKutyNJ. Find NJ.com Yankees on Facebook.