The Bentham prize

By Tamay on Jan. 15, 2020, 6:58 p.m. GMT

We will be awarding bi-weekly prizes of $300, $200 and $100 in Amazon gift cards, as the first, second and third prize to the commenters who make the most valuable contributions to forecasting the questions in Metaculus' Animal Welfare series, in the periods:



Round 1: January 23, 00:00 GMT to February 6 (edit: changed to February 3rd), 23:59 GMT

For round 1, all comments in with the "Animal Welfare Series" category tag (i.e. all of these questions) will be considered toward overall contributions.

Details to follow.

Evaluation Criteria

Public predicting

To be announced.Roughly, a good commenter is someone who contributes to our ability to make accurate and well-calibrated predictions. This might take the shape of a single beacon of good work, or a longer series of contributions.Examples of useful contributions are carefully thought-out public predictions, models, factorisations, comments that help improve questions, datasets, or links to relevant sources.Regularly posting their predictions as comments, and sharing the reasoning that went in to produce it (using the “post prediction as comment” feature that appears after you’ve predicted). Good public predictions are usually accompanied by supporting evidence, analysis or references to other predictions made elsewhere. This 'evidence' may be as simple as "linear extrapolation suggests X", or "betting markets suggest Y", or as sophisticated as developing a Monte-Carlo simulation.

Examples of public predictions

Will the Chinese police or military intervene in Hong Kong in August 2019?



25% notany made a prediction on Aug 14, 2019 at 12:50pm



While the resistance is ongoing, number of people involved has declined. If China intervenes and sends in PAP (The Chinese People's Armed Police Force) they can control the streets but the political and economic disruption may be larger than the riots. More importantly they can't control other forms of less direct resistance that may emerge.



Just to give a some numbers: In 2008 38% of the young (aged 18-29) living in the HK identified as Chinese. In 2019 it's only 3%. Democracy in Crisis: How many free countries in 2028?



82 – 91 AABoyles made a prediction on Dec 16, 2019 at 5:28pm



My intuition is that the world is trending better, and a local downturn in social and political will be resolved within the observed timeframe (much as the 2008 Great Recession wasn't a significant factor in the economic dynamics of 2018). Linear forecasting from Tamay's spreadsheet (which I've modified somewhat) suggests this is broadly true, but the implied forecast (~102) seems much too optimistic (as the community prediction seems to indicate). Accordingly, I place a large component around my gut point estimate (90).



But I imagine I could be horribly wrong, and the world is slipping into a much less free future. Accordingly, I added a wide component in the 80-90 range to keep my score minimally positive in these bad futures.

Models and factorisations

Explicit analytical or statistical models can be powerful tools to guide one's forecasting. Simple techniques such as linear extrapolation can often help make good forecasts. Or simply listing all the relevant factors that need to be considered may prove useful.

Examples of factorisations and modelling

What percent of undergrads at select US universities will be enrolled in introductory ML classes?



holomanga updated a prediction on Aug 17, 2019 at 10:51pm



Linear fit to the data given (done in MATLAB) gives 4.8% to 6.0% in 2019. Will global malaria mortality rates be reduced by 90% when compared with 2015 rates, by 2030?



Jotto updated a prediction on Jun 23, 2019 at 11:19pm



There is a whole pile of factors that one could update on. Some candidates off the top of my head:

Bed nets Resistance to pyrethroids (the insecticide used in the anti-malaria bednets) More housing development in malaria-riddled areas (has a similar effect as the bed nets) Improvements in anti-malaria medications Anti-malaria vaccines using new mechanisms, or tweaks to old ones The general global economy (e.g., a recession could reduce donations for the anti-malaria effort, and a further weakening of local economies in poor places would make it harder to weather the effects of contracting malaria) A gene drive to eradicate the types of mosquitos that give people malaria Changes to funding for the President’s Malaria Initiative, which is a major actor in this space Changes in the proportions between different at-risk groups. For example, suppose if refugees as a group were somewhat more likely to die of malaria than non-refugees, and suppose if five year olds as a group were even more prone to dying from malaria than refugees. Assume no correlation between those two factors (i.e., supposing refugees are equally likely to be five year olds as non-refugees). In that case, a scenario where the ratio between five year olds to refugees decreased would have an expected decrease in malaria mortality. Others It intuitively seems to me that the anti-malaria developments are unlikely to dramatically reduce malaria within 11 years. On the other hand, it seems very plausible that they will contribute to the recent downtrend continuing.

Compiling datasets

Creating and sharing datasets can be an invaluable public good. As such, we hope that our prize will result in you internalising some of its social value.

Examples of dataset compiling

Linking and distilling relevant sources

Examples of useful links

Writing new questions and improving existing questions

The writing of new questions that generate useful insight on existing questions, or related topics are appreciated.Moreover, existing questions can sometimes be ambiguous, or its resolution conditions may be under-specified or unclear. It is useful for this to be pointed out, and to receive suggestions on how the question could be improved.

Examples of comments that helped improve questions

The Contest is open to everybody except except Contest judges

Sponsors reserve the right to cancel or modify this Contest in the event that an insufficient number of entries are received that meet the minimum judging criteria.

This Contest shall be construed in accordance with U.S. law. All Federal, state and local laws and regulations apply.

Winners are required to pay all appliable taxes on the prize and Metaculus assumes no liability for such taxes.

Void where prohibited by law. Metaculus reserves the right to refuse to award any Prize if doing so violates any applicable laws.

All decisions of the judges are final and the selection of Winners is at the sole and absolute discretion of Metaculus.

This is not a checklist. We can imagine awarding the prize to someone who has written only a handful of comments, or to someone who has written dozens of one-sentence comments; to someone who has done only one of the above or someone who has done many. The key determinant will be (our estimation of) the value of the contributions made.For the first round of the contest, the judges will be Daniel Filan, Pablo Stafforini and Tamay Besiroglu.