HalTheWise discusses a factor I missed (until I sneakily edited it in, so you may have read the later version that included it):

One very powerful contributor that Scott did not mention is that in many cases schools are directly or indically intentivized to have a low admission rate. US news & world report released the first national college ranking in 1983, and donors and board members at various schools have increasingly been using national rankings performance, which directly includes low admissions rates, as a measure of how well a school is doing. These rankings and metrics also heavily incentivize having high yield (a large fraction of students that are admitted end up attending) which for a fixed size applicant pool also encourages accepting as few people as possible. This has led to the death of safety schools, because they would rather reject a high performing student than admit them and have them not attend. These factors might also be a driving force behind the rise of common app, since schools are trying to get as many applicants as possible, even if it hurts the quality of their pool.

kaakitwitaasota points out that consulting is an exception to the “where you go to school doesn’t matter” principle: A lot of top firms these days won’t even look at you if you didn’t go to the “right” college. My mother did her MBA at Northeastern, and recently had lunch with an old classmate who ended up at a top consulting firm. My mother’s classmate’s résumé would end up in the trash unread these days–Northeastern isn’t considered good enough.

So while it’s probably true on the macro level that smart kids will do just fine anywhere they end up, there is a subset of extremely prestigious, extremely well-paid jobs which will not even look at you if you didn’t get into the right institution at the age of 18–which, in practice, means that the élite are chosen on the basis of who they were at the age of 14-17. When viewed in those terms, it’s completely nuts.

I’d heard this before; my impression is that a big part of consulting is having prestigious-looking people tell you what you want to hear. If what they’re actually hiring for is prestige rather than competence per se, that could make it a special case

prunesquallor on UCs:

I believe UCs are more competitive because of cost. Personally, I got into a number of strong private schools, and chose a UC because it is significantly cheaper. In the past, this would not have been as much of a factor, because college was affordable. I’m not sure if this applies to public schools in other states. The UCs are the best public schools in the country and are able to compete with high level private schools.

This makes sense, but I’m not sure exactly what the model is. UCs are cheap, so many people across the country apply to them? (but I gave data showing out-of-staters were only 14% of UC students, plus it may not be cheap for out-of-staters). UCs are cheap, so many Californians apply? Aren’t public universities always cheap for people in the state? Maybe cheap UCs mean more top-performing Californians apply to UCs instead of private out-of-state colleges?

BlindKungFuMaster writes about a factor that could explain stories of exceptional students being rejected from everywhere they apply:

It could be the case that college admissions became more random as kids apply more often and the metrics become more vague. I.e. there used to be x acceptable applicants of which 50% were somewhat randomly selected. Now there are 2x acceptable applicants of which 25% are somewhat randomly selected. Then there are just much more kids being unlucky and missing out on all their choices, though it hasn’t really been getting harder to get in. Of course individually the remedy is applying to even more colleges.

This might increase perceptions of selectivity – if one of these bright students posts their sob story, everyone else will just think that standards are very high.

A few people chimed in with concerns about the Dale and Krueger paper. rlms wrote:

From the abstract: “We find that the return to college selectivity is sizeable for both cohorts in regression models that control for variables commonly observed by researchers, such as student high school GPA and SAT scores. However, when we adjust for unobserved student ability by controlling for the average SAT score of the colleges that students applied to, our estimates of the return to college selectivity fall substantially and are generally indistinguishable from zero.” So college selectivity *is* significant even after controlling for student quality as measured by SAT scores. It only ceases to be significant when you also control for some vague measure of ambition as signalled by the average SAT score of all the colleges they applied to.

And reasoner cites a 2009 Overcoming Bias post finding that a previous Dale & Krueger paper on this subject was misinterpreted to say college didn’t matter when it really did matter (Marginal Revolution also covered this). The Dale & Krueger paper I posted was an update to that one that said no, really, college doesn’t seem to matter. But I haven’t had time to look at it closely myself, so this shifts my priors a little bit.

aesthesia is also doubtful:

If I remember correctly, the Dale and Kreuger paper is somewhat limited in its conclusions. Their sample was weighted toward the high end of the achievement spectrum, so it really says something like: conditional on being accepted to Harvard, there’s not much difference in lifetime earnings between actually attending Harvard and instead choosing to attend UC Berkeley. There aren’t a lot of students admitted to Harvard who instead choose to go to the University of Southern North Dakota at Hoople, so we don’t really know what happens to them. This is a somewhat personal issue for me: I read summaries of Dale and Kreuger’s earlier work when applying for college and decided not to bother applying to selective schools, and just went to a middling state university, thinking that going somewhere more selective wouldn’t make a difference in my future life. I’m no longer confident that was the right move. I believe I would have learned more and made better and stronger connections had I gone somewhere more difficult to get into.

But eqdw’s experience bears Dale & Krueger out:

Hello. I work at a major company involved in job search, job ads, hiring, etc. And I would like to share with you something from a quarterly status update I saw the other day. Status update presentation had a slide outlining the results of a user study we did. We surveyed employers to find out what are the most vs least important details they look at when making a hiring decision. Out of something like 20 different options surveyed, “where the candidate went to college” was rated dead last in importance. “Formatting of resume” was rated as more important than “where they went to college” for making a hiring decision. This would seem to confirm conclusion #6 above

Freddie de Boer wants to remind us (another point I stealthily edited into the post later on) that we really are talking about a small subset of institutions here:

I ran the numbers myself several years ago. Out of 3000+ accredited two- and four-year colleges, something like ~150 reject more students than they accept. The large majority accept almost every student who applies. And of course only half of the population will ever enroll in college and only a third will ever finish. The people who say this is a niche problem are correct.

Gossage Vardebian on money:

Certainly the idea of getting into the best school you can, is a further manifestation of the competitiveness discussed above. But that requires another factor, the “and damn the expenses” factor. I went to State U because I wasn’t good enough to get into U of Ivy, but also because I couldn’t afford it. Well, hell, the me of 1980 couldn’t afford 2019 State U either, so hey, might as well go for Harvard too. And if my parents has enough money for State U, well then Harvard is that much less financially onerous, isn’t it? This is also a reason why more kids these days apply to out-of-state State universities, which in turn are admitting more out-of-state kids – the difference in cost is not as large as in the past. It used to be that the idea of going into tens of thousands of dollars of debt for college was insane – literally nobody I know did that. People would do it for med school or law school, and that’s it. I’m sure there were examples from other demographics where that happened, but now of course it is commonplace, as a quick Google search will confirm. Part of this is the whole “you must get into the best school possible” mindset, but part is just a “that’s just the way it is now” resignation. The mental leap to take on that kind of debt has been conditioned into families now.

Anthony adds:

The debt thing is a big change from when I went to college, but it’s driven by policy more than by parents. The government encouraged student loans because those would get paid back while subsidizing tuition at the student end or the college end wouldn’t (directly), and since they were paid back, the had very little budget cost. With a relatively significant college premium, people were willing to take on fairly large amounts of debt (and were not very likely to default). This allowed colleges to spend more, driving up the sticker price, making loans more necessary for students who weren’t poor enough to get direct subsidies or rich enough to afford the higher price. Unfortunately, this is a politically very difficult problem to unwind. Even though the best policy would be to stop subsidizing student loans entirely and make them dischargeable in bankruptcy, which would hugely limit the availability of student loans, that’s never going to happen.

Peffern relates their own experience:

I think point 3.4 is the most important, based on anecdotal evidence. I’m in undergrad at a top school right now. In high school, I was a good student – perfect SAT, good APs, reasonable GPA, etc. Despite having good accomplishments here it didn’t feel particularly effortful – I’m good at math and can structure a coherent argument, so taking the SAT was mostly just getting a good night’s sleep and studying vocabulary for a week. I also did a lot of extracurriculars, and the work and stress load from those absolutely destroyed me. I’m not even talking about “starting homework at 11pm” kind of workload, I mean the social aspect, the cutthroat politics, the status games, and the showmanship. It’s not that I think those things are necessarily bad in and of themselves, but they’re infinitely more difficult than classwork. I’m incredibly busy these days with class and I’m not even on the same order of magnitude of stress as I was in HS. High school students are vicious bastards. When you take the AP calculus exam, or the SAT, or even just the final for some class you take, you are only really competing against the teacher, the test, and yourself. When you do extracurriculars, you are competing against horrible entitled jerks with rich parents who make your life miserable. I would take a hundred AP exams before doing another pointless extra curricular. I don’t know if it explains the college enrollment statistics but it certainly explains the outrage, pessimism, and anger of people my age over the process. I spent what was for past generations an exciting and important time of my life locked in a box sanding off all aspects of myself that didn’t perfectly resembke an ivy league student just to get beaten out by some kid whose father got him an internship at somewhere prestigious. That does things to people.

meh finds reason for doubting my thoughts on the Common App:

The Pew article you link has a different conclusion about the Common App: “The expansion of the Common Application, which makes it easier for students to apply to multiple schools, doesn’t appear to be behind the increase in application volume. The Common App, as it’s called, is accepted by nearly 800 colleges and universities in the United States and several dozen overseas. Of the 1,364 institutions in our sample, 729 accept the Common App along with (or in some cases instead of) their own application forms; the other 635 use their own forms. Although one might suspect that the ease of applying to multiple schools via the Common App would result in stronger growth in application volume among those schools, there was almost no difference in 2002-2017 growth rates between the schools that used the Common App and those that didn’t.”

But adds:

But is their reasoning sound? Isn’t it possible the Common App increases applications uniformly among all schools, not just ones using the Common App? Consider if pre-Common App, I am willing to fill out 10 applications, so I apply to 10 schools. Now with the Common App, say half of them accept it, so I only need to fill out 6 applications. I am still willing to fill out 4 more applications, so I look at my 11th choice. If they take the Common App, I apply to them for free. If not, I fill out an application, and am still willing to fill out 3 more applications. Does this lead to a similar increase in applications for both Common App and non Common App schools?

And alexhutcheson has a more complicated model for why applications per student might have increased:

I think this analysis is excellent, but I don’t think the Common App explains the increase in applications-per-admission. There’s another significant factor that causes students to apply to dozens of schools: No one can accurately forecast what a given school will cost anymore. At some point (I believe in the late ’00s, but could be wrong), elite schools started to extend their financial aid programs to include students from middle-class families. Here’s a press release from Harvard in 2007. Yale and Princeton followed quickly, and most other elite schools seem to have done similar things, albeit with more constrained resources. Prior to this change, a student from a family with middle-class income or above could know with reasonable certainty what a given college would cost: they would expect to pay the tuition, fees, etc. listed on the brochures. After this change, they would have no idea until after they were admitted. They might get a generous financial aid package that brings the cost down to the price of their local state university, or they might get nothing except loans. The systems used to determine financial aid packages are opaque and not well-publicized, so the outcome is unpredictable. In 2009, I applied to a broad sample of 14 schools on the east coast. They were a mix of “elite” private and flagship public universities. My parents were comfortably middle-class. I was lucky enough to get into most of them, and so I had the opportunity to compare financial offers. In maybe 1 case out of 12, I would have had to pay the full sticker price. The rest would have been heavily “discounted”, but the discount varied widely between schools, from ~5% off the total cost of attendance, to 60% off, to 100% off (full ride). For the private universities the “discount” came from a mix of “need-based aid” and merit scholarships, while for the public universities it was exclusively from merit scholarships. In this system, you don’t know what the financial offer is until you get in, there are possible windfalls from getting a generous financial offer, and it’s difficult to predict in advance what the financial offer will be until you get in. The incentives here are obvious: student who are conscious about the cost of their education have a strong incentive to “play the lottery” by applying to as many schools as is feasible for them, while biasing towards schools that are known to provide generous financial packages.

faoiseam says that APs might be a good objective way to track the increase in competition over time:

The College Board has lots of data on AP exams since 1996. They track how many people take exams, in what grade, by race and result, which is enough to get a sense of how people did. They also keep track of AP scholars. In 2002, 1738 people were National AP scholars, (8 APs with score 4 or more) enough to get you into HYPSM. This increased to 7k by 2005, 16k by 2010 and now is 35k, not enough to get you into a top 20 school. By this measure, college entry has got more competitive. The number of schools offering APs has increased in that period, from 10k in 2000, to 17k in 2010, to 20k now. Looking at individual subjects, the two most taken are AP Lit, and AP Calc. 5s in lit increased 10 fold from 2000 (6978/105k) to now (61k/580k). For AP Calc BC in 2000 we have 5k 5s from 15k exams, in 2010 39k 5s, from 79k exams, and now we have 56k 5s from 139k exams. As these are the most common exams, this shows that while the number of schools barely changed in the last 10 years, the number of 5s has doubled. In general, things have gotten twice as hard since 2010 (as measured by AP exams), and 5 times 3005, and 10 to 20 times since 2000. The AP curriculum has stayed pretty stable, so that does not explain the change. The number of schools has been pretty steady since 2010, though it did double since 2000. This does not explain all of a factor of 2, as the 10k schools that offered APs in 2000 were the better schools anyway. This makes me question why the US does not used AP exams for college admission. If, as I estimate, the number of people who get n 5s doubles as n increases by 1, this suggests that four 4s and four 5s should be enough to get into HYPSM. The UCs should requires 8 4s. or 7 4s and 1 5, etc. This could be done by computer when AP results come out, delaying admission until the Summer of senior year. It would be really nice to have a clear bar that was required for entry, as opposed to the current opaque system.

Steve Sailor fills in a hole in my data:

I wrote a column back in 2013 documenting that famous colleges were not expanding their freshmen class sizes at anywhere close to population growth over the last generation: Consider the growth rate of Harvard, the world’s richest university. The number of undergraduates in its class of 1986 was 1,722. After a quarter of a century, during which the US population grew by 75 million, Harvard’s class of 2011 was 1,726: an increase of four. This is not to say that Harvard isn’t expanding: Faculty and grad students are up, and non-teaching staff skyrocketed. Similarly, Yale’s undergraduate student body has been the same size since 1978. Five years ago, the second-richest college announced a proposal for adding a couple of dormitories, but construction won’t proceed until another $300 million is raised. In 2010, MIT unveiled plans to expand undergrad enrollment by six percent, which would only get it back to where it was in the 1990s. Among the most prominent colleges, Princeton is the only one over the last generation to have actually succeeded in boosting enrollment (and that by only about ten percent) after it opened the Whitman residential college in 2007.

…and gives some fictional evidence:

Robert Heinlein’s 1950s juvenile novels are full of scenes where people ask the hero what he’s going to do for college and he says, “I dunno, I guess the day after Labor Day I’ll drive down to State and sign up for some classes.” (That was pretty much what UCLA was like when Heinlein went there for awhile in the 1930s.) But in the end of “Have Space Suit, Will Travel,” the hero gets accepted into MIT when the head of the CIA, or whatever, pulls some strings.

This concept of heading down to the college the day it opens to sign up for classes reminds me of an anecdote I rejected for the main post because it was too unbelievable. Wikipedia quotes Supreme Court Justice Byron White saying he went to Yale for law school because he was planning to go to Harvard, but got sick enough on the train to Boston that he decided to get off at New Haven and go to Yale instead. But I haven’t been able to find any source better than Wikipedia for this, so I don’t know if it’s true. Does it at least mean that this was plausible enough in the 1930s (when White was going to college) to make a believable urban legend?

graeme on HackerNews gives the Canadian perspective:

You may be interested in the view from up North. Joseph Heath, a Canadian academic has commented on how the top three Canadian schools teach more students than the top ten American schools. I’ll excerpt: ————- “But of course there’s a reason that it’s so difficult to get into Yale – it’s because Yale has only 5,400 students, in a country of over 315 million people! By contrast, McGill has over 30,000, and University of Toronto has 67,000 undergraduates, serving a country of only 35 million people. That means there’s roughly one spot at Yale for every 58,000 Americans, compared to one spot at McGill for every 1,100 Canadians. No wonder American life is more competitive. Furthermore, all of the best schools in the United States are tiny. Here is a list of the top 10, as ranked by U.S. News & World Report, along with the number of students (undergraduate, I believe): Princeton: 5,336 Harvard: 6,658 Yale: 5,405 Columbia: 6,068 Stanford: 7,063 Chicago: 5,590 Duke: 6,655 MIT: 4,503 Upenn: 9,682 CIT: 997 Dartmouth: 4,193 That means the top 10 universities in the United States – a country of over 315 million people – at any given time are educating a grand total of only 62,150 students. By contrast, here are the rough numbers of undergraduates at the top 3 Canadian universities: McGill: 30,000 UBC: 47,500 UofT: 67,000 So the top 3 Canadian schools are at any given time educating a grand total of 144,500 students – more than twice the total of the top 10 U.S. schools. (In fact, the University of Toronto alone has more student capacity than the top 10 U.S. schools combined.) The United States has almost exactly 9 times the population of Canada, so in order to have the same sort of capacity in higher education, the top 27 schools in the United States would have to have 1.3 million students.”

This suggests a model where people only have so many cognitive real estate devoted to remembering what the good schools are, and so if your top three schools are very large, you can have many people in the “cognitive elite”, but if your top three schools are small the elite will seem more selective. Seems like a stupid way to run an educational system 🙁

jumpinjacksplash offers a model of how an arms race would work:

I think what people may be missing is that time is a thing: In a world where there are 10 places at Harvard every year but 9 applicants, everyone who can read Latin gets in, and no-one bothers to study harder than what it takes them to meet the minimum requirement. In a world where there are 10 places at Harvard but 11 applicants, the worst Latin-reader doesn’t get in. But someone who realises they’re the worst Latin readers will practice more, and someone else will be the worst. That person will then practice enough to make someone else be the worse, who then practices a bit more. Given enough time, the standard keeps rising as everybody needs to stay ahead of everybody else; eventually everyone hits the ceiling of Latin literacy and starts taking up the violin, then competes on violin-skills until they all have to start climbing Everest. The only equilibrium is where everyone exerts maximum effort on top of their abilities. It takes a long time for this to become even remotely necessary (I still don’t think we’re close to being there). But a world where the first 18 years of your life consists of maximum effort to get into college is a dystopian nightmare, hence as we trend towards it everything gets constantly worse. This also explains the anecdotal people who expect to swim into Berkeley but get rejected from UCLA: their parents think they’ve massively overshot what’s required, but they’ve failed to realise how far down the slope we’ve slid.

I failed in my original goal for this piece, which was to present an account of competition getting tougher across society. There wasn’t enough data, so I fell victim to a streetlight effect where I concentrated on colleges, and then another streetlight effect where I concentrated on Harvard (which is the best-documented college). But the reasons why competition is getting tougher at Harvard probably don’t generalize, so I ruined my chance to have a really good post about modern competition. Still, a few people stuck to the original spirit and discussed increasing competition as it was showing itself in their own field.

anechoicmedia from the subreddit on TV:

This reminds me of a depressing perspective relayed by Brady Haran on podcast. An acquaintance had described getting started in the TV business back in what would probably have been the 1980s or 90s. Like most young people of the pre-computer world he arrived with no independent film making experience. As he describes it, he presented himself to a major TV station, applied for a job, and was accepted into an apprenticeship of sorts, in which they taught him all the ways of TV-news-making from the ground up, which he then went on to excel at. Brady, himself a BBC veteran, remarked that such a thing could never happen today. Entry positions are ferociously competitive. Young adults applying today can be expected to have had a quality camera in their hands since childhood. There are no apprenticeships; New hires are expected to show up with fundamental knowledge already in hand. But more than that, the modern ability to signal your ~passion~ via independent YouTube/Vimeo presence means you are really expected to show up with an entire history of volunteer video production going back years. How can you be worthy of working in the BBC if you haven’t been using the tools at your disposal to produce content for years already? You must not really want it — you’d be laughed out of the office without an extensive list of “extracurriculars” in hand. The ladder of opportunity had been yanked quite a few rungs upward. Something like this may also be at work with the expectation that “real programmers”, especially younger ones, have many side projects, open source contributions, etc.

And jaghataikhan on consulting again:

This reminds me of my field (management consulting). It’s pretty competitive to get into the big three (McKinsey/Bain/BCG), so the level of preparation in applicants is also a runaway arms race. One of my buddies who went to MIT said he’d known people who’d prepped for case interviews for two years! In contrast, I remember seeing some pre Recession recruiting videos at my firm where there was a girl from Yale saying something to the effect of “I didn’t even know what management consulting was before I interviewed!” I can’t tell you how quaint/laughable that sounds vs. the extreme levels of competition for the same jobs today. Basically if you don’t have a top resume/school/marks/extracurrics/internships/cases/etc from day 1 (and still a matter of luck btw unless your dad is a partner or something), don’t even bother applying. And I’m certain the level of competition is skyrocketing daily.

KingWalrax gives the only thing I’ve heard that could possibly be a general explanation of the phenomena, though I don’t know if it’s right:

Ex-YCombinator President Sam Altman wrote a decent essay explaining how Zero-Sum Existential Competition & Conflict arise from a lack of baseline Growth in any given system. Whatever you think of Tech/YC, it’s a good read and Sam is a smart guy and he’s on-the-money here. GDP Growth is always spiky/noisy, and highs average out with lows, making it difficult to discuss overall performance. But Sam’s graphs tell a pretty decent picture: we haven’t had a year of 10% GDP Growth since the 1980s, but our recession-year performance (i.e. 2008) is as bad as ever. Your essay here already tied the broader economic landscape in to College a few times (Farming & Sources of employment in pre-1800s New England). It strikes me as relevant again here in this post-WW2 era. Growth declined. Therefore: Zero-Sum Competition increased. The only other option was individual actors settled for lower individual outcomes, and you wrote the best essay out there on why that doesn’t happen.

I’ve seen conflicting data on whether long-term growth has really decreased and when that started, and you can see some discussion of this in the comments. If it has, it would explain a lot, and would mean there’s still room for things to get worse.