Sometimes the world needs you to think new thoughts. It’s good to be humble, but having low subjective credence in a conclusion is just one way people implement humility; another way is to feel unentitled to form your own belief in the first place, except by copying an “expert authority”. This is especially bad when there essentially are no experts yet — e.g. regarding the nascent sciences of existential risks — and the world really needs people to just start figuring stuff out.

Over the years, I’ve had a vague impression that many educated people somehow shy away from forming their own impressions of things, other than by copying the conclusions of an “expert authority” or an affect like “optimism” or “pessimism”. And they often think it would be foolish to even try anything else.

Note: If you’re unfamiliar with subjective credences as probabilities, This Tutorial should help. If you’re uncomfortable assigning subjective credences to things, try The Credence Calibration Game.

Obstinance: not what I’m aiming for

Before I go on complaining, remember the Law of Equal and Opposite Advice. There was a dark time when people felt it was reasonable to argue about when Michael Jackson was born… not to figure out the answer, but to win a debate about it. Some people reading this might be too young to remember, but there was a time when folks would get into heated debates about the population of Canada. Party lines would be drawn, sometimes never to be crossed again. People were obstinate.

Alice: “I still think it’s 20 million.”

Bob: “Whatever, I know it’s 24 million. I heard it on the radio the other day. Charlie knows I’m right.”

Alice: “Come on Charlie, you heard the radio. It was 20, right?”

Charlie: “No, no, it was 24.”

Alice: “Screw you guys.”

This is obviously suboptimal. They should be pooling their evidence, not fighting over whose is better. This is an instance of “Everyone is entitled to their own opinion” gone awry, and it’s not what this blog post is about. This post is a correction to a correction.

An example: personal health

One’s personal health is another topic where, like existential risk, there are no expert authorities. There are doctors, who have reasonable priors that are based on averages of people with a similar symptoms to you, but are not necessarily well-specialized to you.

I’ll use an example to illustrate what I mean here — not sufficient evidence to convince you of a trend, but hopefully enough to point at what I’m talking about: the reaction I get to using a binary search to identify which dietary supplement was most likely to have good short-term effects on me, where I found that a mixture of B vitamins, prior to knowing what any of the supplements in particular were supposed to do, had the strongest noticeable positive effect on me among a bunch of other supplements.

Folks often respond to my description of the experiment with “Couldn’t that just be a placebo effect?”

This puzzled me. The placebo effect wouldn’t favor me liking any particular supplement over any other, given that I hadn’t known what to expect from them in advance.

Then they’d say “That’s just anecdotal evidence”, as though somehow probability/statistics couldn’t enter into the reasoning process. This also puzzled me, until I realized that a lot of people only think of probabilities as measuring frequencies, and aren’t aware that they’re also a very robust way of tracking and combining uncertainties that follow nice coherence laws.

Then they’d say “Couldn’t it just be confirmation bias?”, which again, wasn’t a strong hypothesis given my lack of expectations…

But as the conversations continued, I started to realize that what they were actually saying is that science is hard and you can’t do it on your own.

There is some truth to this. There’s a lot to consider when designing an experiment, and it helps a lot to have some other eyes on your design. But I’m worried that this attitude is going a little too far. Maybe stories about “biases” and “placebo effects” and the like are causing smart people who are actually capable of statistical inference to… just give up. Give up and let the scientific establishment publish articles about average effects across populations. Don’t try to figure out stuff on your own, unless you’re part of a lab wherein the establishment has given you the role of figuring out something in particular… then, you’re golden.

What’s going on here

This is, of course, an exaggerated description of a problem, but nonetheless, I think a problem exists here to some real degree. At least a good 20 or 30 people I could list view “biases” and “placebo effects” like magical sources of wrongness that can’t be modeled, accounted for, or guarded against, except by somehow by some authoritative group called “experts”. Why is that?

Again, I’m not hinting that we should give up on teamwork. I’m saying that you can use probability, too. If you can do math and hold facts about the real world in mind at the same time, it’s not that hard.

So why not try?

It seems to me that what it actually takes to do statistical inference on your own, for people who already have the mathematical and empirical fluency, is cajones. Or, to use a less etymologically sexist term, a sense of entitlement: Entitlement to figure stuff out for yourself; not to be a nay-sayer who says “There’s no evidence” for this or that, but to stick your neck out sometimes and say “I’ve seen enough evidence now for me to believe X… bring on your criticisms and counter-evidence!”

Qiaochu Yuan makes the observation, “I think people fail to realize that when you’re self-experimenting, you don’t necessarily want to science hard enough to establish conclusions approximately true for everyone: you’re already happy to establish conclusions that are approximately true for you. And you just don’t have to science as hard for that.” I theorize that, moreover, that failed realization comes from failing to see science as a probabilistic inference procedure — like the kind you’d use to guess the color of the next jellybean from a bag — and instead viewing science as a Thing That Someone Out There Is Doing Properly. Compare “The last 8 [jelly beans | times I tried the medication] have all been [green | successful], so the next one probably will be, too”, and contrast with “You can’t do studies on yourself ‘properly’, ergo you shouldn’t do them at all.” This same attitude hampers research into existential risks: “We can’t run a randomized controlled trial across parallel Earths yet, so there’s no data to inform safety measures yet”. That may sound like an exaggerated stance, but I recently heard Andrew Ng, a prominent machine learning expert, literally telling a crowd of around 1000 researchers, “Existential risk from artificial intelligence is like overpopulation on Mars. […] No one can predict more than 5 years into the future, so it’s really not the time for responsible scientists to be thinking about these things.” … as though you needed to know a building was definitely going to burn down before you’d install a fire extinguisher. (Luckily, a panel moderator called him out on this.)

World-scale relevance

What worries me most about this is not that folks might not figure out what supplements to take, but the effect on people reasoning in general about what the world needs. Too many potentially-highly-effective do-gooders, upon realizing that there are facts about whether certain world-scale interventions are effective, might fear the embarrassment of being wrong so much that they’re afraid to do their own analyses.

This is bad. Granted, what I’m saying is a higher-order correction — a boosting, if you like machine learning — to the important first order realization that there are facts of the matter about what you can do for the world, which I think is even more important. But charity evaluators like GiveWell can’t figure everything out for you. In fact, GiveWell started because not enough people were doing careful analyses of charitable causes, and that’s still a problem today. Condorcet’s Jury Theorem requires some amount of independence in the population for the group as a whole to find the truth… just like how independent measurements with different instruments is a much better way to confirm a physical phenomenon.