This is the second of n many result posts for the second survey I hosted. If you missed part 1, click here, also check out part 0 on perceived and real political identification.

For today, we’re covering just a single section, the one about evaluating intellectuals. For this, I chose 10 people, several of them in the IDW, and asked everyone to basically rate their ability to make sense on a 1-10 point scale.

This might beg the question: who cares? Isn’t that super elitist? Why spend an entire section on this? Why not ask different questions instead?

The reason is that if you want to have accurate beliefs about a wide range of topics, then the most important skill is to identify which people to listen to. In today’s world, there is some public figure out there who is a legitimate expert and usuallky correct on most topics, among many other figures with far worse judgment, who have a following anyway. Take economics, for example. Is every economist correct? Impossible: they say contradicting things. Is no-one correct? Hard to imagine. But then there must be some correct ones among a fairly large group. Identify them, and you can have a more accurate set of beliefs than the average expert. In a sense that’s unfair, but that’s how it works (with some caveats, you won’t understand the reasoning as well etc.). People don’t usually think of this as a skill, but it totally is.

Alright, so the next set of questions was to rate each of those 10 people (you’ll see who they are on the first chart). Then I asked everyone which of those 10 they’re unfamiliar with, which would also undo their previous rating (might be useful since the survey doesn’t let you un-click radio buttons). Then I asked a question about which of the 10 are deliberately and regularly lying.

Let’s do familiarity first.

Question 19: Which of these people are you unfamiliar with? “Unfamiliar” means you either have no idea who they are or have only heard them mentioned in passing. (In particular, this will cause any answer you gave above to not be counted, so if you clicked a button by accident, this will effectively un-click it.)

I “won” a prediction on this; I predicted with 50% confidence that Julia Galef would have the highest degree of unfamiliarity. But I didn’t expect the the results to be this extreme.

So two quick biographies: Julia Galef co-founded the Center for Applied Rationality and she is the host of the Rationally Speaking Podcast. She also did a Ted Talk. Eliezer Yudkowsky founded the research field called “AI Alignment,” the Machine Intelligence Research Institute, the rationalist site LessWrong, and he wrote the most reviewed Harry Potter Fanfiction of all time. He appeared on the podcast in episode 116.

I’ve heard people say that there is a big overlap between r/samharris and r/slatestarcodex, but I don’t think it can be that big given these numbers, right?

Question 18: For each of the following people, please rate how impressed you are by them as intellectuals, where 1 is a total nincompoop, and 10 one of the best thinkers we have. Please remember that it’s not about how ethical they behave or how likeable they are. You may skip anyone you don’t know well enough or don’t want to judge for any other reason. If you think they’re actively lying to their audience and so the scale doesn’t apply, please leave it blank (there’ll be a question for that later).



Here are the mean results:

But hold! I mentioned that I would not count answers from respondents about people they say they don’t know. Okay, well, but that’s not gonna make any noticeable difference, right? Let’s see how many responses I deleted and how many remain…

It turns out that literally over half of all respondents voting on Jula Galef, and over a third of the respondents voting on Eliezer Yudkowsky, have done so without knowing who they are. How much does it influence the results? Quite a bit.

More than a 1 point swing! And a 3 rank swing! (For Julia.) Now that’s some bullshit. Imaginge if I had not asked that question – after all, I did it primarily as a way to offer a pseudo-unclicking mechanism. I didn’t expect it to have a big impact. We would then have seriously flawed results for a profoundly stupid reason.

In some way this is great, because it’s a more interesting (certainly a more surprising) result than anything I had hoped to get at with this section. But it’s also total bullshit.

So Julia has suffered significantly more from this than Eliezer. Is this sexism? Are people who don’t know anything just assuming she can’t be as smart because the name is female-sounding? Looks like a plausible conclusion, given that these people presumably had only the name to go on.

Let’s hope you have not immediately answered this with a yes, because a look at the deleted answers shows that there is no (or almost no) evidence for sexism. The people who have rated both of them despite knowing neither have usually given them the same score, with only a couple of exceptions. The difference is explained solely or almost solely by the fact that Eliezer has more actual responses, so the BS ones have a lower relative impact.

I’m going to have a look at the respondents who gave 1s to people they don’t know at some point. I’m curious what they are like.

So, why is this happening? This is an apt time to mention another statistic. Despite my repeated reminders that no-one has to do the entire survey (I mentioned that on all three reddit posts I advertised it with), 252 out of 277 people have replied to the Ham Sarris question in the final section. And 265 to the quesiton about X-risk form AI in section 6. So almost no-one actually skipped large chunks of the survey. It seems like people just tend to answer any one question, even if there is absolutely no reason why they should. That is the best explanation I can come up with, at least.

So. Have the corrected ranking of intellectuals.

Ah, but a simple bar chart is boring for a linear ranking! Instead, check out this totally awesome graphic that I made! The numbers stand for Mean / Median / Variance.

Some labeling problems here, I know. At least for Dave Rubin, there’s plenty of space. Oh, and speaking of Variance, let’s plot them by variance.

Jordan Peterson is the winner here; that’s not surprising. Yudkowsky is number two, also not surprising. The top three have the lowest variance, that also makes sense.

I (barely) lost a prediction here. I predicted with 80% confidence that Sam Harris wouldn’t be #1 on the evaluation. But, even after deleting the BS ratings, he’s still 0.03 in front of Richard Dawkins. That’s too bad… though, admittedly I didn’t even have Dawkins in mind when I made that prediction.

I’m surprised Ezra Klein is this low. I remember his podcast being mentioned favorably on the sub several times. Related: I forgot about one prediction in part 1; I predicted with 50% confidence that Lessons of Death would in the top 5 most picked for favorite episode, but I “lost” that prediction; it was only picked twice. This was also based on stuff I’d seen on reddit. Takeaway: stuff I see on reddit may be less representative than I’d thought.

Another thing I looked at was the number of #1 and the number of #10 ratings.

(4 people think the smartest person in the world is a 1. Whelp.)

If you’re like me, you’re now wondering about the one guy who thought Dave Rubin was a 10. What were his other ratings? I went and I checked, and he literally gave Ezra Klein a 1 and everyone else a 10 (he’s the only one who’s done something like that – I guess he just really hates Ezra Klein.) He also claimed to know everyone, which seems pretty unlikely. I then went through his individual responses looking for an excuse to delete them, but I couldn’t find it. Seems like he did the rest of the survey for real. Alas, congratualations: out of the 10 240 000 000 000 possible responses to questions 18 and 19, you’ve successfully picked the one that would do the highest amount of relative damage to Ezra Klein’s results, and now you’re getting away with it, because I don’t want to delete results without a more objective reason.

Something else: if you recall, the last question from part 1 was about the favorability of Jordan Peterson. That has been from this section, and the reason I put it there was because I was curious about how strongly it correlates with the replies on how much sense JP makes when he talks. In theory, liking someone is different from thinking they’re smart, so these two things could come apart, but I would still be surprised if the correlation was significantly worse than perfect. Let’s see…

… yeah. So this is only the people answering [*] on the X-axis, and the mean score on Peterson on the Y axis.

Question 20: Do you think any of these people are actively, regularly, knowingly lying to their audience? If so, please list the names of all for whom you think it applies. Otherwise, please leave this blank. (I’ll evaluate this by hand, so you don’t need to be careful with syntax).

I’m not sure what I thought when I decided to evaluate this by hand, but it was most definitely a mistake. The bar I set seemed quite high, but it got a lot of responses regardless. There’s inevitably a bit of freedom here, because some people didn’t just list names like they’re supposed to. I tried to count people who said “maybe Shapiro” half; so for every two that did, I counted one. If you wrote something more nuanced, it probably wasn’t counted, but I’ll list your replies in a bit.

Some other responses:

Shapiro and Rubin are obvious hucksters who pander. Ezra and Sam play to their audiences, but are more intellectually honest. Ezra will always take the most in-vogue PC position, Sam plays to being edgy and won’t back down on naive geopolitical views. Peterson has a schtick, but sometimes will reevaluate his positions, and tries to keep his views ambiguous/obscure rather than commit to a position. He’s somewhat internally consistent once you know which areas he’ll obscure.

Ben Shapiro uses his language instrumentally, not honestly. Dave Rubin is not a liar or a bad guy, but he is a ding-dong.

Ben and especially Dave are intellectually dishonest.

Rubin, probably Shapiro, Peterson when he’s referring to why Marxists won’t debate him

All of them

(Maybe that’s because you don’t know Eliezer Yudkowsky and Julia Galef? Sorry dude, “all” as an answer doesn’t fly if you’re telling me you don’t know everyone. And it’s too easy to just write “all,” anyway. List them.)

I question of ben shapiro believes that he is acting in good faith. I also question if jordan petersen avoids critiquing trump because he doesnt want to lose patrons. Not sure if this meets that bar you set.

Shapiro might be lying. Rubin looks like a psychopath with only his own interests in mind. Peterson is both deluded and out to delude people (perhaps, because he thinks it’s for their own good).

All but Dawkins, Harris, Yudkowsky. Brett doesn’t lie much, but plays heavy apologetics because of who he is usually talking with. Dawkins can work himself into a froth about certain things that seems a bit weird and perhaps untruthful. I don’t think Klein lies on purpose, he seems to want the truth always, but I haven’t listened to him enough to be sure.

(You contradicted yourself; also you said you don’t know Julia.)

all

(You also don’t know Julia.)

Ben Shapiro – Yes, in the form of debate club style arguments to further his political positions that aren’t necessarily fully genuine. Dave Rubin – Yes, due to thirsty mugging and profound audience capture. Ezra Klein – Yes, at least in the form of shading the truth in the maintenance of a good liberal facade.

(This was unambiguous, so I counted it.)

Ben Shapiro treads a dangerous line, by cowing to Trump supporters (aware his base has skin in the game).

Shapiro and Peterson stand out, the others can be misleading (charitably)

Now we get to the most important question.

Question 21: (Bot / Troll detection): the earth is…

You will notice that “flat” doesn’t appear in the pie. Kudos to us: we managed to have a survey with 277 responses where 0 picked “flat” on this question, even though it was the first option. I think this is clear evidence that people should stop worrying about the flat earth movement and start worrying about the giant lizard shape movement instead.

Others:

Roughly spherical

Spheroid, according to google

an oblate spheroid

an oblate spheroid

A squished sphere

Spherical

Orbiting the sun (I’m a self aware bot)

It’s an oblate spheroid – but only 0.0034 off from a perfect sphere. https://chem.tufts.edu/answersinscience/relativityofwrong.htm

Spherical

Spherical

Oblong Spheroid

(this is actually a word!)

Obloid spheroid

oblate spheroid

nice 🙂

Spherical.

A sphere

oblate spheroid

Spheroid

Spherical

spherical

Banana shaped

Pear shaped

kinda round, but quite rough once you really LOOK INTO IT

It’s closer to an egg shape

PRAISE SOBEK!

oblate spheroid

I don’t have much to add here. Let’s finish with comments about this section.

Douglass Murray and Charles Murray are odious as well

Please help

Earth is spherical

I hope I don’t get ‘The Earth is round’ wrong.

I wanted to say ‘a disc atop four massive elephants riding a giant turtle’ but I might be disregarded as a troll 😉

More people would be nice

(I know, but people are complaining about length already…)

Missing some obvious people in SH’s intellectual orbit; Majid, Ayaan. Missing other well known “new atheists”, Daniel Dennet comes to mind. Also some of Sam’s regular antagonists, like Reza Aslan.

I’m sad Omer Aziz wasn’t on here so I could give someone a 1

This section is a 4 but it was nice to be reminded of Julia.

Once you’ve clicked a radio button you can’t unclick it. Meaning I would be stuck rating someone if i clicked on accident.

(That was the point of asking you whether you know them? I even said so.)

The earth is a spheroid :p

Your bot question is bad

I was afraid selecting ‘other’ on that last question would (wrongly) identify me as a troll, so I’ll just leave this here. The Earth is located on a giant turtle, with other turtles ‘all the way down’.

It doesn’t matter if they’re lying or not. The outcome is the same.

(*In Sam’s voice* I believe that intentions matter.)

No

There should to be the ability to undo a bullet entirely not just change the response.

What’s Yudkowsky doing here

(I made the survey so I get to choose what interests me :P)

ok but what if it is shaped like a giant lizard

i wanted to write flat I really did

(Glad you did not; that would have ruined the record.)

If I had answered that the earth is shaped like a giant lizard, which is absurd, how would that indicate that I’m either a bot or trolling?

(No; I decided not to discount any answers based on this question.)

Due to how awfully stupid Dave Rubin is, everyone else’s grades got a boost of at least 1.

Why are there questions about other people than Sam?

(Because people other than Sam are also interesting.)

The earth is shaped more like a potato

Calling out liars feels a bit… weird. I have a negative opinion of several of the people on that list, but I don’t know (enough to say) if they are lying to their audience.

I think Shaprio is not an idiot, but does have horrible opinions

“Do you think any of these people are actively, regularly, knowingly lying to their audience?” question I assume refers not to the people immediately above (since I’ve just stated I don’t know them), but the ones further up.

(Yup.)

Do bots really fill these out?

(Probably not. I added this on a whim and was curious so I left it in.)

The Earth *is* shaped like a giant lizard, though.

If I put “shaped by a giant lizard” will my vote still count?

(Yes.)

And that’s it! Part 3 will cover Politics and perhaps the Faith and Philosophy section aswell.

∴ ∴ ∴ ∴ ∴