(CNN) It was supposed to be a landmark moment in Serena Williams' stellar career: Being crowned Sports Illustrated's Sportsperson of the Year.

But the somewhat begrudging reaction to the 34-year-old's accolade bears witness to America's complicated attitude towards gender and racial equality, according to one horse racing expert.

Williams became the first black woman to win the prestigious award after trumping an unlikely rival to the prize -- thoroughbred racehorse American Pharoah.

Both have an impressive pedigree in 2015.

A 21-time grand slam winner, Williams lost just three of 53 matches, winning three major titles, while American Pharoah won the hallowed American Triple Crown series and the Breeders' Cup title.

Serena vs. American Pharoah: Who should have won Sports Illustrated's award? https://t.co/5dR2tIQeTg #SISportsperson — CNN Sport (@cnnsport) December 16, 2015

But there is simmering resentment that despite attracting 47% of the vote in a SI's readers' poll on who should win, Ahmed Zayat's colt was overlooked for the award itself.

The readers' poll has no bearing on the actual award, but is mostly a way of encouraging reader participation in the process.

But respected British racing journalist and broadcaster Brough Scott is in no doubt the right recipient has been chosen, telling CNN: "I don't think there is any comparison between the two.

"The poll is fascinating -- if that is how Americans are reacting, that is something of a wider discussion about society. It is about their attitudes to race and female emancipation -- I don't see how you could view it any other way."

Out of nearly 600,000 votes in the Sports Illustrated online poll, Serena collected just over 5,000 votes (about 0.9%), while American Pharoah romped home with over 275,000.

"There has been an anti-Serena element because she didn't fit the stereotype of the old-fashioned, elegant white female tennis player," added Scott.

"She was big and muscular and black. Let's be candid about it, there's been plenty of that sort of unspoken prejudice against Serena, I'd have thought, over the years."

Dear Sports Illustrated, Why did you bother to have a contest & hold voting for "sportsperson of the year" when... https://t.co/gAExDLnk30 — Victor Espinoza (@EspinozasVictor) December 14, 2015

Williams said she was "beyond honored" to become only the third individual female winner in the award's 61-year history, saying it offered "hope to continue on and do better."

But while her delight was obvious, frustration from fans of the four-legged fraternity was equally evident.

No racehorse had completed a Triple Crown victory in 37 years and American Pharoah followed that up by winning the $5 million Breeders' Cup in the final race of his career.

One Twitter user wrote: "American Pharoah overwhelmingly won online poll and yet you pick someone else? What a joke."

Brian Zipse, editor of Horse Racing Nation, claimed SI had an "agenda" while the horse's jockey Victor Espinoza called it a "sham." Zayat was not immediately available for comment when contacted by CNN.

The magazine's editor Christian Stone explained part of its decision to honor Williams lay in "the grayer, less comfortable ether, where issues such as race and femininity collide with the games."

Williams' decision to end a 14-year exile at the Indian Wells tournament after claiming she suffered racist abuse in 2001 proved her "a conciliator seeking to raise the level of discourse about hard questions," said Stone.

Congrats @SerenaWilliams on being named @SInow Sportsperson of the Year! Girls everywhere are dreaming bigger because of you. -mo — The First Lady (@FLOTUS) December 14, 2015

SI's criteria also demands a winner that embodies "the spirit of sportsmanship and achievement," which might have proved a decisive setback for American Pharoah.

Broadcaster Scott agrees, arguing that part of the problem is that some in racing circles tend to anthropomorphize horses. At the end, he says, they're animals.

"I yield to no-one in my admiration and affection for them, but revel in them for what they are -- don't try to pretend they are humans," Scott said.

"I am a great believer that the kindest thing to do if a horse fractures a leg is to euthanize it as soon as possible. It has no imagination but as an animal it can feel pain. To try and put it through endless and often useless attempts to patch the leg up ... You've got to take its life.

"People say 'how can you do that?' but the fact is horses don't come here by stalk, they come here by man's arrangement. We bring them into this world and it's our responsibility to take them out.

"I'm not going to have them rot away if they are no longer able to function in the way they feel happy."