One of the most polarizing books out there is Nicholas Nassim Taleb’s The Black Swan. Ostensibly a book about the role that luck plays in life, I’ve always found it to be much more than that. Yet some people say the entire book is for idiots. What gives?

Well, this is actually true. But once I threw my teabag into the sink and my teaspoon into the trash. If you take a moment to reflect on the latest stupid thing you’re embarrassed about, we can all agree that booster shots against idiocy should be prescribed widely.

I can’t compress the entire book adequately, so I won’t try to. I’m just going to talk about one big component of the book, from my perspective, and maybe show you how far this perspective can take you.

And if you know anything about Taleb, you’re possibly thinking “We’re taking that lunatic seriously?”. Yes, bear with me. This is a piece about what I think Taleb is saying, not a piece on what you should think about Taleb as a person – but he’s too funny not to bring up from time to time.

If you don’t know anything about Taleb, oh boy are you in for a treat.

I. Narratives Make Bad Points Sound Compelling.

Here’s the message Taleb has that I want to discuss. You don’t know that much, you can’t possibly know that much because you’re just a human, and the least you can do for yourself is to be clueless with your eyes open. Some things are just too complicated, but we are great at convincing ourselves we know what is happening. And one of the main ways we do this is telling stories.

A large amount of the book could be summarized as:

“Structuring your theory into a story makes it more compelling.”

Everyone knows that. You knew that, right? This is the most common piece of advice I saw given to students, and I still give it out. Write a story. Facts we can take or leave, but a narrative always sells.

At first, this sounds pretty harmless. It’s just a way of communicating clearly. You start at the beginning, move on to the middle, and finish at the end. Get the hell out of here, Taleb, nobody needs you.

And wipe that smug goddamned grin off your face.

That’s pretty much exactly what I thought the first time I read the book. Gave it a read, chucked it on a shelf, told everyone it was garbage.

Then I spent some time at university and started to notice some things. It started in my third year, right around when I had picked up The Black Swan again on a whim.

I saw a TED talk by psychologist Paul Piff with 3.5 million views, titled “Does money make you mean?”. Here’s a quick summary (in his words) of the study:

“I want you to, for a moment, think about playing a game of Monopoly. Except in this game, that combination of skill, talent and luck that helped earn you success in games, as in life, has been rendered irrelevant, because this game’s been rigged, and you’ve got the upper hand. You’ve got more money, more opportunities to move around the board, and more access to resources. And as you think about that experience, I want you to ask yourself: How might that experience of being a privileged player in a rigged game change the way you think about yourself and regard that other player? So, we ran a study on the UC Berkeley campus to look at exactly that question. We brought in more than 100 pairs of strangers into the lab, and with the flip of a coin, randomly assigned one of the two to be a rich player in a rigged game. They got two times as much money; when they passed Go, they collected twice the salary; and they got to roll both dice instead of one, so they got to move around the board a lot more. [trendy man talks some more] And here’s what I think was really, really interesting: it’s that, at the end of the 15 minutes, we asked the players to talk about their experience during the game. And when the rich players talked about why they had inevitably won in this rigged game of Monopoly … They talked about what they’d done to buy those different properties and earn their success in the game.”

Wow, stop the presses! We’ve figured out why rich people are awful!

(How much do seats at a TED talk cost anyway? Maybe this is a bad place to flex, Paul … wait, are they applauding? Christ, I have no idea what this says about society, but this is above my pay grade.)

It turns out all your personal biases were totally right! Everyone doing better than you is a jerk. It’s so simple! Wealth makes you into a monster.

Only the blood of the innocent can sate my perverse hunger!

“What we’ve been finding across dozens of studies and thousands of participants across this country is that as a person’s levels of wealth increase, their feelings of compassion and empathy go down, and their feelings of entitlement, of deservingness, and their ideology of self-interest increase. In surveys, we’ve found that it’s actually wealthier individuals who are more likely to moralize greed being good, and that the pursuit of self-interest is favorable and moral.”

“And that is why I am pumping chlorine directly onto the most expensive seats!”- Paul Piff, probably.

There are plenty of technical reasons to doubt this result, ranging from statistical objections, to simply believing that most scientific findings are false. But there’s a much simpler red flag.

I wish I had this next revelation because I was brilliant, but it was actually because I was nineteen and liked being a contrarian. But think about this for a moment, really stop and imagine this– how stupid would you have to be to get double the money in a Monopoly game and get to roll twice as many dice as your opponent, and still think you won because you kick ass at Monopoly?

Really, stop and think.

If the answer is unbelievably stupid, congratulations! It turns out it actually is unbelievable. If you didn’t get this answer, please see me after class.

Gregory Francis, a professor in the U.S, spends a whole bunch of time drawing attention to dubious studies, mostly by working over their statistics. He is, to use the modern vernacular, a total dork. However, we forgive him, as he goes through some pretty basic statistical concepts and demonstrates that Piff’s studies are turning up positive findings way more frequently than they should, even if we assume the effects are real. He just did the math and said “For an effect of the size you are reporting, your method for detecting it would fail about half the time. Why are all your results positive?”.

(As an aside, Francis has previously accused authors of running tons of tests and failing to report the non-significant findings. One accused Francis of checking tons of authors and failing to report the innocent ones. The scepticism ouroboros has formed and the epistemology serpent has eaten its own tail.)

Anyway, in Piff, we have an author in a field that famously has trouble replicating studies, proposing an implausible effect, with strong evidence that something is wrong with the studies coming out of his lab. I have no idea what he did wrong, and it’s entirely possible he has no idea either.

I told some of my friends (who are incidentally some of the top graduates in the country now) that something suggests there’s an issue with the study, and asked them what they thought it was.

Please, sir, we know you know where this is going. Restrain yourself.

Ah, the problem is clearly that the sample size is too small! Ah, the problem is that Monopoly does not generalize to the real world! Ah, the problem is the sample was too homogenous (this one was pretty funny because I didn’t say anything about the participants – they just assumed it was a valid critique because the university runs so many studies on undergraduates).

Whether or not these critiques are valid, no one just said “Come on, that’s just silly. People aren’t that obnoxious. They can be pretty obnoxious, but there’s an upper limit.”

You see, I have no idea whether money makes people immoral. Maybe it’s even likely that it does! I’m a leftist. Have you seen how you behave when you do anything brag-worthy? But that’s irrelevant right now, and best handled in therapy.

No, the real point is that the claims from this study are ridiculous and intelligent people that have been studying for years can’t pick up on it. The real point is that I am actually really confused as to how Piff got his results, but at least I’m not tricking myself into thinking I know what’s going on. Did he fudge the numbers? Was the experiment poorly set up? Just pure bad luck in sampling? Hell, is the result true? I don’t know, but I will advance the position that you can’t either.

The problem with narratives is that they make everything sound convincing even if they really shouldn’t. The problem with this result is blindingly obvious – a twelve year old would object to your victory under this ruleset, and the objection would be sustained. But we stick a little story on here – “The ridiculous result makes sense because…” and suddenly it gets past intelligent people.

Narratives are indiscriminate. Attach them to anything and they are now more compelling.

A useful way of thinking about persuasive techniques is how Scott Alexander frames them as either asymmetric or symmetric. An asymmetric weapon is only helpful if your point is correct – you will get frustrated quickly if you try to use the scientific method to prove that the earth is flat. However, speaking in a clear voice and being very handsome is symmetric. It helps whether or not you’re making any sense. Narratives are symmetric.

II. Narratives Are Incredibly Prevalent.

We’ve seen that bad results can be hidden in plain sight in the context of psychology by simply slapping a narrative on these. But does this only apply to psychology?

I’m going to demonstrate that it happens everywhere – sorry, psychology-hating fans, we only focused on psychology because it’s what I know best.

So there are a huge number of things you can’t possibly understand. Sometimes this is because you don’t have enough time to do a PhD in them. Sometimes it’s because no one understands them, although we frequently just slap a label onto a field and say we understand what is happening.

Incidentally, Taleb insists on using the word flâneur, but once nicknamed a guy he hates ‘Fortune Cookie’. You can’t make this stuff up.

Like, there’s the usual stuff about who won the primary debates in the U.S. Ignore the politics right now – I’m picking this example purely because it is recent and clear. After each debate, we get to hear who the ‘winners’ probably are. After the second debate in early August, it was opined that a candidate did really well. I’m not American and have no idea who this is. I don’t care. But here is what I do care about.

“Tulsi Gabbard: The Hawaii congresswoman entered this debate as one of the least-known candidates in the field. That should change — at least somewhat — after a strong performance. Gabbard was reasonable but also pointed: She did real damage to Harris on criminal justice reform. She was poised and knowledgeable throughout. And she made the most of the relatively limited talking time she had, using it to talk about her resume — most notably her service in the Iraq War. Overall, a very strong performance.” CNN

Cool, cool. I wonder how big the change was –

Wow! Literally five whole people changed their minds! It’s almost like nothing you just said matters!

People are reading these and thinking they know what’s going on. At the very least, doesn’t the person writing so confidently seem like they think they know how the electorate works? I haven’t even watched any of the debates. I needed an example, and I knew their prediction would be silly. I just looked up ‘post debate winner’, picked the first article from a major news outlet, then the first instance where the writer made a testable prediction. It’s like clockwork.

Did I mention there is a goddamn webcomic about Taleb? Has society gone too far? What are you saying, old man?

Yet while this stuff would fly constantly in some instances, it was clear that this didn’t happen during my machine learning classes. Why not?

Well, if you did something stupid, you would get really poor accuracy on the task at hand, and then you’d be the guy that is bad at building models. The reason it’s easy to get away with in other fields is that if you make the story up after the fact, it’ll always fit whatever you just saw. Then to prove you wrong someone would have to get funding, ethics approval, all that jazz. So it’s easy to make up stories, but hard to disprove them.

That seems like a pretty good explanation.

But as I’m writing this, I wonder, have I not just engaged in the very thing I am rallying against? How do people get away with awful political takes and economic forecasts? You can test those. Why isn’t anyone calling them out? Does no one care? Are they getting called out and I just don’t know where to look?

It’s an interesting question. But at least the person from CNN above actually made a testable prediction. I just hope that the lesson they took away was “I actually have no idea how any of this works”, rather than the far more common outcome of coming up with a story to explain why they were wrong.

In contrast to this, FiveThirtyEight famously produces very accurate predictions. And when they look over the actual results following events, they tend to avoid narratives. They fit some trendlines and test some straightforward hypotheses. And here is FiveThirtyEight’s own head honcho Nate Silver apologizing for how all the stories he was spinning made him totally wrong about the previous Republican primary. There’s nothing to be ashamed of there at all because at least he admitted it instead of making excuses. It happens to everyone, but only a few people can just admit they were clueless.

The truth is, making predictions is hard. People are complicated. The future is opaque. The world is a mess.

That is, making accurate predictions is hard if you’re interested in being right, or if you have an outcome you could be punished for. However, coming up with compelling narratives? Super easy. There’s a very famous study about ‘power poses’, again a popular TED talk. It just so happens that one of the authors has retracted the belief that the effect is real, but the one that gave the TED talk has strongly defended the original work (unconvincingly) and is still raking in the fame. RetractionWatch says that the author is receiving large speaking fees, though following their link I couldn’t find a specific number, so I’m not sure about that. Either way, it seems like there’s no real punishment for pushing bad science.

And there’s no shortage of these weird papers:

“How many times have you heard the expression – maybe you’ve used it yourself: I’m going to wash my hands of something. I’m going to move on, I’m going to disassociate myself from something, I’m washing my hands of this. And this idea of symbolically washing yourself clean is something you see in religion too, of course, like the ritual of baptism. But could there be a little more to the metaphor? Could washing your hands, the act of methodically lathering up with soap and water, have some tangible effect on your thoughts? A study out this week in the journal Science suggests that the answer is yes, that hand washing can actually change your thinking.”

How do you control for variance in grandmothers?

Yes, this study did not replicate. But I’d like to draw your attention to something else entirely. What would have happened if the results came out the other way? My guess is that we would have seen an interpretation like:

“We proposed that the act of washing one’s hands reminds participants of uncleanliness. Washing one’s hands in Western culture is heavily associated with negatively valenced acts, such as ‘washing one’s hands of a crime’.”

If you can come up with a compelling narrative regardless of your results, it seems like the narratives are actively damaging in almost all cases. But there is some inherent tension here, because it seems like establishing a causal relationship between factors requires a narrative. Things fall when they are dropped because of a force called gravity – so we can’t just stop using all sentences that have the word ‘because’ in them if we want to draw conclusions. All we can do is try to figure out is when it’s appropriate to do so.

III. The Inappropriate Use of Narratives is Enforced in Academic Contexts.

We have seen that inaccurate results can be hidden in plain sight by appending a short story at the end. But I’ll go even further and say that we are inadvertently training people to think this way, punishing them when they don’t, and even this process is hidden by narratives.

I mentioned earlier that many of the students I know have a rote list of allowed objections to science. This is actually not really their fault – it’s one of the unspoken rules at university. The main reason I was an effective private tutor is that, not being employed by the university, I was allowed to speak the unspoken rules.

I noticed that the university would tell my students (and myself) to do independent research and write up our conclusions but simultaneously tell us what story to tell.

And you had better go with that story – or any story.

So you get an essay about psychoanalysis, with a little blurb saying that ‘evidence shows it is approximately as effective as cognitive behavioral therapy’, plus a couple of articles laying out a simple narrative.

No, no, you don’t have to accept that as true. This is a university, critical thinking is encouraged. But if you don’t, you’ll get marked down and we’ll say your research was weak. What constitutes weak research? Well, you didn’t have enough references (ignoring the fact your peers had far fewer). Oh, you had more references than them? Then you should have elaborated more on fewer articles. Ah, you did both? You’re over the word count now.

How could you have known before submitting this what was too much or too little? Oh, we’re out of time, lodge a complaint if you have any further issues.

I wish more of my students had thought to ask that question. How could you have known before submitting the work? For many students, the answer is that you probably couldn’t. There’s a bunch of randomness in marking, and it depends a ton on how your tutor is feeling (and yes, on the quality of your writing). But not on the actual facts of the matter in many courses. There is typically an allowed narrative, and woe unto those who stray too far from it.

Pictured: That tutor your hate, probably.

Do universities have a sinister agenda to force particular studies down your throat? No, not usually. Does this apply to every unit? No. But it does to most units, most of the time. One of the key pieces of feedback I got on my previous article where I mostly discussed psychology was that every faculty has some version of this problem.

See, you do a bit of digging, and hey, it turns out one of those first references wasn’t quite right. But you know that you have to include the starting references to make sure you’re on topic, plus it’s expected of you. Anyway, you have to cite the article that you’re refuting.

If you get this far, you’ve already screwed up. The mistake here is that you’ve gone and included something that isn’t a narrative. Sometimes studies aren’t false for compelling narrative reasons. The researchers don’t always make a really interesting mistake, or interpret a fact incorrectly. Sometimes you just recruited too many outliers into your sample. It happens.

If you write “Research thus far has been inconclusive. John & John found a positive effect, but a later study by Bill indicated a negative effect.”, you’re already at a disadvantage. That’s just confusing. I need closure!

You would be far better off saying “John & John found a positive effect, suggesting that this therapy has become considerably more effective in light of recent cultural shifts.”.

The former demonstrates more research. The latter is a neat story and is easier to read. Don’t complicate things. Just let me read your work and put a big red tick next to it. Do you think your underpaid marker working through their 30th essay actually gives a damn about the total amount of research you did? In fact, do you think they even know why they’re giving you a precise number? Why 81% instead of 80%? Why 80% instead of 79%? At the end of the day, there’s going to be some gut feeling involved, but they’ll come up with some reasons you can’t argue with after the fact.

Why does this exist? He even talks like Taleb tweets.

In all major Australian psychology courses, students are bell-curved to ensure too many people don’t qualify for Honours. Please find the obligatory horror stories here. When a tutor accidentally assigns too many good grades across the class, they are asked in no uncertain terms to pick someone at random and come up with a reason to take marks away (I have insider information on this point). They do it every semester, and the students have no way of telling whether they’ve been given the shaft. The reasons sound great, whether or not they were concocted after the fact.

I once had two students, one of whom got caught plagiarizing about a quarter of their submission, and the other turned in perfectly serviceable work. Same assignment, same semester, same institution. The one that plagiarized scored about 10% higher.

If you’re a student, dear reader, this is the system that you’re putting in charge of your self-worth. Don’t make that mistake. It’s just numbers on paper. Higher numbers are helpful, but they aren’t worth getting depressed or anxious over.

Can you see how utterly bizarre this is? The university provides students a narrative about the state of the literature. Students are expected to respond with a narrative, or will be marked down because lacking a narrative simply is less enjoyable reading. The markers will come up with a story to justify the assigned grade, which at some level is a function of how happy they were with lunch that day. At no point is the epistemic and emotional damage that this entire process inflicts upon students addressed.

A final, non-psychology example. I have no serious opinions on sociology. I haven’t studied it nearly enough to comment on the field as a whole. However, I did take a unit on sociology that was really stupid, along with some very stupid units in other fields. At the end of it, they asked everyone to write a reflection on what they learned. I think most people have had to sit through this kind of nonsense. But we all know that we have to diligently write down all the trite ways in which the unit radically transformed our views on life.

And that’s really the key. You can get a good grade by saying the unit was amazing in beautiful prose. You can get a lower grade by saying the unit was amazing with less beautiful prose. You can get a bad grade if you complain that the unit wasn’t that good.

If you say that everything in the unit was just post-hoc rationalizations that don’t really explain anything? That’s not playing the game, not even poorly. That’s just flipping the table over, and you’re going to get sent packing with a failing grade. It might sound fun to point out that the emperor has no clothes, but do you know the thing about emperors? They can decide to hang you and no one will stop them. But if you play along?

Of course, your highness, robes of the finest silk. Anyone can see that..

IV. In Conclusion.

I’ve pointed out how some bad studies can be covered up by narratives. I’ve pointed out how this goes further than that, and can turn up on major news sites. And finally, I’ve pointed out how even the university system implicitly enforces this method of engaging with the world and defends itself using it. There are many other frameworks, each with some truth to them, that one can view these problems through, but this is the one that resonates with me the most.

There are plenty of problems with Taleb. I don’t deny that. No one could possibly deny that.

But he was the first person that got me seriously interested in statistics, and insofar as I have ever been effective, it has been because The Black Swan managed to convey its message in a way that made it feel relevant to my life. I view everything through the lens he has given me – and it has some downsides, which maybe I’ll cover some other time, but it has no doubt been a huge positive impact on my life.

This may not actually be for good reasons. The tone has been described as profoundly arrogant, but maybe that’s the way to get a message through students who think they’re that much better than everyone (hint: you aren’t). At least he was funny.

But I owe him one anyway, and I think most people would benefit from reading his work. Even the fact he repeats himself so frequently, I view as more of a feature than a bug. Whenever I’ve had a big change in my perspective, I had to go through a period of viewing everything from that framework, and that never really goes away. But I think people generally need it hammered in pretty hard before it actually occurs to people to enact these things in a day-to-day settings, as opposed to just discussing them at dinner parties with other nerds.

With that, I believe I’ve said what I wanted to say about Taleb. He is possibly a terrible person, or a great person, or a genius, or an idiot. I have no idea. I doubt he’d care what I think because I like his biggest idea and he’s rich. But I feel like I should leave on a more appropriate note.

Hm. Perhaps an anecdote about – No. Ah. Of course.