On Tuesday I watched Evgenii Shvetcov from Russia win the Men's 400M T36, narrowly beating the man I was cheering on – Team GB's Paul Blake – into second place. But any disappointment I felt suddenly disappeared when, seconds after crossing the finish line, Shvetcov looked into the TV camera and used sign language to communicate a message to people watching back home.

This may seem disloyal to these shores, but if I'd known Shvetcov was deaf, I'd have been cheering him on from the start. There are deaf athletes who have competed in the Olympics and Paralympics at London 2012, but they're few and far between. Another separate disability is required for them to compete in the Paralympics because, remarkably, there is no category for deaf athletes.

In this country, deaf athletes have for a few years now been the poor cousins in funding terms. The Guardian's Datablog identified total spending of £11bn on London 2012 (£9bn from the government, and £2bn budgeted by Locog), while UK Sport has been spending more than £100m per annum on Olympic and Paralympic sports and athletes. In the context of those sums, £42,000 seems minuscule, but it's the amount of funding that UK Deaf Sport used to receive annually from the government. When this sum was withdrawn in 2008, with then sports minister Gerry Sutcliffe claiming this was to focus funding on athletes competing in the Beijing and London games, it put deaf athletes' participation in the Taipei Deaflympics 2009 in jeopardy.

If you're wondering what the Deaflympics are, then take a look at Tuesday's BBC Ouch! interview with Craig Crowley, the president of the International Committee of Sport for the Deaf. As the name implies, the Deaflympics are for deaf athletes; they were first held in 1924. Not only do the Deaflympics use adapted technology to suit deaf people (think visual cues, like a light being used instead of a starting gun, or a referee's whistle being replaced by a flag), sign language is the prime method of communication among the majority of athletes and spectators. Crucially, the Deaflympics bring together athletes who have faced similar communication barriers, and allow them to compete on a level playing field.

While the Olympics and Paralympics enjoy massive funding, the Deaflympics has a much lower mainstream profile. The games in Tapei in 2009 were a success, but in 2011, disaster struck when the 17th Winter Deaflympics in Slovakia was cancelled at the last minute, with the sole organiser accused of embezzling £11m.

There are hopes that the upcoming 2015 and 2017 winter and summer games will get the Deaflympics back on track, but on the biggest stage of all, at London 2012, deaf athletes have once again been mere spectators. When you see the success of team games featuring blind athletes or wheelchair users, it's hard not to wonder why we can't watch, for example, teams of deaf football and hockey players competing against one another. Deaf footballer Claire Stancliffe told me: "We missed out on London 2012 in many ways. The facilities are fantastic, the support for GB athletes was out of this world and it was an opportunity for deaf sports to be showcased to millions. It would have changed people's perceptions."

Crowley has pointed out that since 2001, Paralympians have enjoyed access to the same venues, transportation and technology as Olympians. However, Deaflympians have not been offered the same opportunity. He has also gone on record as saying that he believes "the only way to gain visibility for deaf athletes is to work alongside the Olympics and Paralympics", and he hopes that deaf sport "will be the surprise" of the Rio 2016 Olympic Games. However, the process towards achieving some level of deaf sports at those games is complex. For example, Crowley told me, because the International Paralympic Committee has "no mechanism to bring in a new disability, they would need to add a new classification to allow for deaf athletes".

In the meantime, the scarcity of deaf athletes at the games represents a massive missed opportunity. One of the rare exceptions is Laurentia Tan, a profoundly deaf equestrian and London 2012 Paralympic Games silver and bronze medal winner who also has cerebral palsy. However, another athlete I saw showed how deafness isn't taken into consideration at the games. The British athlete Olivia Breen, who has cerebral palsy and came fifth overall in the 100m T38, also wears hearing aids. When she didn't hear a false start being called in the first round, the rest of the field stopped, but she continued sprinting to the end of the track. Fortunately, the race was postponed for an hour to enable her to recover before running again, but it's hard to imagine this happening at a Games that included deaf athletes and was more aware of their needs.

Over the past few weeks, we've seen triumphs by sports men and women from across the world, from a wide range of backgrounds, of abilities and disabilities. Let's hope by the time the next Games come along, more athletes with signing hands, hearing aids and amazing lipreading skills are able to write – or sign – their own compelling narratives for the world to see.