This past week, I was named as one of the 30 Under 30 Developer in Canada. While It’s an honour to be recognized as such, I’ve noticed a gaping hole here: I can likely name 30 Canadian developers under the age of 30 who are objectively better programmers than I am. This has prompted me to wonder: how could awards be improved?

The awards process is fairly straight forward — individuals are nominated by their peers every year, and the top 30 are selected from the set of nominees. One might argue that as long as the evaluation of nominees is fair and accurate, any deficiency of the list of selected candidates must be attributed to the limitations of the nomination process itself, and the shortlist it produces.

While it would be a valiant endeavour to study how we could ensure a fair selection process, I will focus this post on a series of possible solutions to this nomination problem, so that a fair and accurate judgement may produce an increasingly accurate list of the top N individuals in a given field, under certain age, geographic, or other requirements.

Lack of Awareness

One fairly obvious problem with awards is that many in the desired population don’t even know the award exists, or that nominations are being accepted. In my case, I hadn’t even heard of this award before I was informed I had been nominated, however naturally the awareness about a given award scales with the length of time it has been in existence. Everyone knows when the Oscars are (having existed since 1929), and since almost all eligible films will be considered, a fairly accurate list of nominees is usually produced (ignoring their final selections among these cohorts of nominees, which are sometimes subject to biases or irregularities).

Naturally a solution to this problem would include a focus on marketing and promotions encouraging people to nominate their peers, however another method of raising the public’s awareness of the award nomination process could be to improve the accuracy of the award’s results. This is based on the assumption that as an award becomes more accurate, people become more interested in it (due to some internal desire to survey human success or to maintain a reputable source of knowledge about excellence in a particular field). While this interest in human excellence is common in some of the arts, as evidenced by the numerous well known awards in film and music, the scientific community rarely experiences the celebrity status that actors and musicians presently experience.

Famous people just doing famous people things at the 2014 Oscars

If we did accept the premise that increases in the accuracy of the award result in increases in the attention paid to it, we would notice a cyclical relationship, whereby improvements to an award’s accuracy result in a rise in public awareness, thereby improving the nomination process and once more improving a list’s accuracy. All else unchanging, awards will naturally converge on accurate lists over time, becoming more accurate as time goes on. This helps to explain why awards which have existed for longer durations are generally accepted as more accurate representations of human excellence.

Nomination Incentivization

Now that we’ve moved on from the problem of awareness, we must ask the question: just because the public is aware of an award, what can be done to maximize the frequency with which an individual will nominate their deserving peers?

Most awards offer some form of bounty for winners, in an attempt to incentivize individuals in the field desire to be included. Some prizes like the Nobel offer large sums of money, others like The Oscars offer inclusion into an elite club (the Academy of Motion Picture Arts and Sciences), but almost all awards offer some fame to their recipients. We notice that the more attention an award receives, the more recognition the winners will receive, leading more individuals to strive to be nominated, once again improving the pool of nominees, and the accuracy of the list.

However besides incentives for the recipients, there remains a question of whether peer nominators should be incentivized. For instance, awards could post bounties ahead of time for nominations, which only pay out to nominators if their entry is chosen. This could be further improved by requiring submissions to contain at least N nominations (paying if at least one of them was chosen), which would encourage a more rigorous study to discover deserving nominees, rather than simple one-off nominations of friends. Nonetheless it remains to be seen whether this would actually improve the quality of the nominations, or if it’s better instead to have people nominate each other exclusively of their own volition.

It is also possible to disincentivize nomination, by requiring nominators to deposit a stake on behalf of nominees, which is fully or partially slashed or returned if the nominee is selected. Alternatively, one could temper this disincentive by splitting the relinquished stakes among the nominators who were correct, turning the nomination process into something of a prediction market. While it’s worth mentioning that this staking-for-entry-to-list system could begin to resemble a Token Curated Registry, I believe a TCR would actually be more beneficial in solving the problem of nominee selection rather than that of nomination itself, so I’ve omitted a more in-depth analysis of how TCRs could be used to curate awards.

Nominees as Nominators

Drawing from my own experiences with this particular award, one fairly obvious opportunity does show itself, for awards to somehow engage nominees themselves to continue suggesting new nominees for consideration. This would take advantage of subject matter expertise, whereby those who specialize in a particular field are best equipped to make judgements about further excellence in that field.

One version of this could be requiring nominees in a given year to submit a list of nominees who they believe should also be considered for the award. By requiring this list to be just a fraction of the total number being selected N, nominees can have some confidence that they won’t necessarily lose by recommending those whose excellence surpasses theirs. Nominees would also be interested in making sure the list is as high quality as possible (while still including them), since their excellence is recognized among an echelon of those on par or superior to them. Assuming that people are incentivized enough to nominate others who are more deserving than they are, such a scheme would eventually converge on an increasingly accurate list of nominees, as more and more excellent people are added until none remain.

Another version could be the nominator bounties mentioned above, with the added restriction that only past nominees (as recognized subject matter experts) are eligible to fulfill the bounty.