Story highlights "It was almost like he was doing baseball signs," says deaf actress Marlee Matlin

Deaf Federation of South Africa says man was not a recognized, professional interpreter

DeafSA says the man used "meaningless" hand signals

The four-hour Mandela memorial was broadcast to millions worldwide

In this case, the pictures apparently weren't worth a single word.

The sign language interpreter at Tuesday's memorial service in Johannesburg for Nelson Mandela may have appeared to have been translating spoken words into gestures during the four hours he appeared on television screens around the world, but he was a fake, observers said Wednesday.

"The so-called 'interpreter' ... at FNB stadium has been dubbed the 'fake interpreter' and the deaf community is in outrage," said Bruno Druchen, national director of the Deaf Federation of South Africa (DeafSA), in a statement.

"He is not known by the Deaf Community in South Africa nor by the South African Sign Language interpreters working in the field."

The man showed no facial expressions, which are key in South African sign language, and his hand signals were meaningless, Druchen said. "It is a total mockery of the language," he added.

The service to commemorate the statesman, who died last week at 95, was broadcast to millions of viewers.

While dignitaries addressed the crowd at Johannesburg's FNB stadium, the unidentified, suited man produced a series of hand signals that experts said meant nothing.

"It was almost like he was doing baseball signs," deaf actress Marlee Matlin told CNN on Wednesday, through a sign-language interpreter. "I was appalled."

Though each country has its own sign language, all of them entail facial expressions, she said. She called his lack of facial expression "a giveaway."

"I knew exactly right then and there that he wasn't authentic at all, and it was offensive; it was offensive to me."

Others also took offense.

On Tuesday, during the man's performance, a deaf member of the South African Parliament and vice president of the World Federation of the Deaf, tweeted that the man "is signing rubbish. He cannot sign. Please get him off."

South African sign language interpreter Francois Deysel tweeted: "Please can someone ask the interpreter to step down from stage, it is embarrassing and making a mockery of our profession."

DeafSA said the man also did not use the established, recognized signs for Mandela, South African President Jacob Zuma or his predecessor, Thabo Mbeki, among others.

"This proves that he is not involved in the deaf community and doesn't know South African Sign Language," it said.

"To the best of our knowledge, he has not undergone any formal training in South African Sign Language or Interpreting offered by any recognized institution which offers these training courses."

As outrage over his interpretation skills grew, so did questions over who he was and who hired him.

A spokesman for the ruling African National Congress said the party had not hired him for the event.

"We have used him on some occasions but yesterday was not an ANC event so we cannot answer for yesterday," spokesman Jackson Mthembu said.

The South African government was investigating the reports, said Collins Charbane, minister for performance monitoring and evaluation in the presidency.