In a classroom in Bronzeville, on Chicago’s South Side, eight students are locked in intense debate about Lawrence Kohlberg’s stages of moral development. They’re tearing Kohlberg apart, with justification, as far as I can tell, but keeping up with fast-paced Socratic dialogue about complicated philosophy is not my strong suit. I’m visiting this college, Shimer, because something quite calamitous has just happened to it.

The communications officer, Isabella Winkler, gives me a tour. Which lasts about three minutes. Shimer is tiny. The entire college is squeezed onto two slightly disheveled floors rented from a more successful neighboring college – the Illinois Institute of Technology. There are no sports teams at Shimer, no sororities. This place will never get ranked America’s No1 party school (which is currently the University of Pennsylvania, according to Playboy). No: the list Shimer currently tops is a miserable one. The reason why I’m here is because it has just been ranked the No1 worst college in America.

So what’s it like, this worst college? What criteria put it there? The compiler, Ben Miller, a former senior policy advisor in the Department of Education, explained in the Washington Monthly that they were looking for colleges that ‘charge students large amounts of money to receive an education so terrible that most drop out before graduation.’ Actually, Shimer topped a list that was adjusted for race and income. So a truer description is that it’s the worst college in America that doesn’t have many students of color or low-income students.

The thing is, in the hours after the rankings were published in October, something unexpected happened. Whilst almost nobody stepped forward in defense of the other colleges on the list, Shimer fans began vociferously attacking its inclusion. One graduate wrote that it’s ‘a totally unique snowflake, and comparing it to other schools is next to impossible’. He added that he wouldn’t take his time there back ‘for all the money in the world. Looking back, I’d go into even more debt to make sure it happened.’ Another called it ‘an eccentric little school that appeals to few; but for those few, it evidently serves as a valuable and stimulating harbor.’

What is Shimer, I wondered, this barely heard-of college that’s so statistically terrible – the very zenith of terrible – and so loved? One comment I read was especially intriguing: ‘I’m unemployable, maybe, and debt-ridden, but it was worth it.’ I flew to Chicago.

Shimer College was founded in 1853 in Mount Carroll, an Illinois Prairie town. They’ve been battling various catastrophes for decades. The local train service was shut down in the 1970s, making their first campus untenably isolated and also – according to the New York Times – ‘a haven for drug users’. That nearly finished them off. But they scraped enough money together to move to Waukegan, Illinois in 1979, and now to Bronzeville. They offer only one core program, and just one teaching method. This is a ‘great books’ college. The great books of the western tradition, not the professors, are the teachers: Da Vinci’s Notebooks and Aristotle’s Poetics and Homer’s Odyssey and de Beauvoir’s Ethics of Ambiguity and Kafka and Derrida and Nietzsche and Freud and Marx and Machiavelli and Shakespeare and the Bible.

Textbooks about the great books are forbidden. That would be too easy. It is primary sources only here. Students can concentrate on humanities, or natural sciences, they can take electives in feminist theories, or Auden, or Zen masters, but it’s all great books and nothing else. There are no lectures. Each class takes the form of Socratic dialogue between the students, guided by a professor if necessary.

According to the Huffington Post, Shimer was the second smallest college in America in 2012, after the Alaska Bible College. In its 1960s heyday it had 400 students. In 2011 there were 126, which slipped to 112 in 2012, then down again to 97 in 2013. And now they have 74.

“You see the trend,” says Isabella, the communications officer.

They’ve been working hard to reverse it, she says, by improving their presence on the web and “implementing alumni volunteers for recruitment. And it looks like the numbers for spring are finally showing an upswing.”

When I ask Isabella to describe to me how she describes Shimer to others, she gets flustered: “We spend a lot of time thinking how to sell ourselves. What do we say? I do marketing but I’m not a professional. I’m basically a lapsed academic.”

Right now, on their website they brand themselves as ‘very small’ and ‘focused’ and ‘rigorous’, and geographically close to ‘endless archives of unadulterated information’. They make themselves sound like a monastery. They also say: ‘We count among us prodigies who left high school early, homeschoolers, transfers, veterans, sci-fi writers, multi-linguists, painters, philosophers, contrarians, misfits, and the double-jointed’.

Out in the open lounge students lie scattered around. One of them – Jibran Ludwig – recounts to me the strange tale of how he ended up here. One night two years ago, when Jibran was 15, he was sitting in an Amtrak station in upstate New York reading Plato’s Republic and getting annoyed by the translation. The Greek gods had been given capital Gs, which struck Jibran as an erroneous Christian reinterpretation, and so he decided to abandon the book and try and find something closer to what Plato had intended. It was election night. The woman sitting opposite Jibran had an iPhone. So he went over to ask her the results.

As he approached her, his Plato in his hand, she looked alarmed. “Oh,” she said after a moment. “I thought you might be one of my students and I didn’t recognize you and I felt embarrassed.” She explained that she was Susan Henking, president of Shimer, “and Plato is the kind of thing we read”.

“If the president of this college thinks I might already be there,” Jibran suddenly thought, “maybe it’s where I should be.”

After that night, Shimer was all Jibran could think about. And now, two years later, he’s made it. They accepted him without a GED certificate or school diploma, which is lucky because he hadn’t gone to high school. He lived in an eco village in Missouri called Dancing Rabbit and his education consisted of him walking around the village asking people to teach him. Some of them agreed, he says, but a lot said no because they were too busy with administrative meetings, and Dancing Rabbit has a lot of administrative meetings.

“The day after I arrived,” Jibran tells me, “I was sitting here, surrounded by people I could relate to, talking about interesting things, and I just noticed that I felt completely at home.”

There’s Kara, who was homeless before coming here, living out of her car for a while. ‘I was lucky enough to get a Shimer pamphlet in the mail in middle school,’ she later emails me. ‘I carried it around throughout high school. The idea of reading books all day for credit was very appealing to a girl who was constantly in trouble for reading when she wasn’t supposed to. It was the only school I applied to.’

None of the students I speak to seem especially concerned about the worst college ranking. But the teachers are. This is serious for them. They’re struggling to stay afloat. Faculty members are taking pay cuts. A ranking like this might tip them over the edge.

“Just today I got another Google alert about it,” Isabella says. “It keeps getting picked up by other news organizations.”

“What was the mood like at school the day it was published?” I ask her.

“Panic,” she says, “a little bit, I would say, in the leadership.”

“This is a small place,” the college president Susan Henking tells me. “If we lose it, we lose a genre. And it’s fragile. So I got upset.” She pauses. “Do I think I should ignore those numbers? Obviously not. I don’t like that ranking and I hated being at the top of one where my values are very centrally located, around underrepresented people’s ability to go to college. My God, I hated being at the top of that.”

The Illinois Institute of Technology in Chicago, on whose campus Shimer sits. Photograph: B O'Kane/Alamy

Everyone here agrees that it’s unjust to reduce this strange, tiny, madly intellectual place to statistics. Isabella says that since 2008 they’ve lost between eight and 10 students per year, for various reasons like health, family, finances. One dropped out because they were offered a job as a professional musician. Even Ben Miller, the list’s compiler, seems remorseful that Shimer topped his list. ‘I think their story is at least partly due to small sample sizes,’ he emails. Then he reiterates this twice more in other emails to me.

Saying all that, I do sense a bit of a race issue here. One student, Katya Schexnaydre, tells me that studying the great books can be irksome if you’re not white because so many of them are inherently racist, and if you’re the only person of color in class “you can feel like a cultural representative. You don’t necessarily want to lead the class on something when you’re not an expert in the field.” I can see why some people might find that dispiriting and transfer out for that reason. (It would be wrong to imagine Shimer as a wholly white community, though. I see lots of students of color during my day here.)

Shimer wouldn’t be a college for me. All that Socratic dialogue would bewilder and exhaust me. I’d want somewhere more practical. Plus it would be nice to have a couple of clubs to join. And so, as the day progresses, I feel myself side with those who bailed on the place, propelling it to the top of that unhappy list.

But then – just as I’m about to leave - something happens to make me change my mind.

I get talking to Albert Fernandez, a professor of cultural history and humanities. He has the intense demeanor and indeterminate European accent of a Slavoj Žižek. He leads me into a classroom as austere as he is and tells me how angry he is about the list.

“What we do at Shimer,” he says, “is difficult. It’s difficult to sit in a small room with six or eight students and have your beliefs challenged. If a school is hard to graduate from for reasons to do with an attempt at educational quality – that should be taken into account. The writer said nothing about that.”

A look of fury crosses his face, at the thought of Shimer being penalized for what makes it great. He says a lot of places that top those best colleges lists are the opposite of difficult. They’re undemanding. “If you’re going to take education seriously you can’t have a system where the objective is to make it as easy as possible to get through.”

Later, when I get home, I spend a few hours watching YouTube videos of graduation ceremonies from hugely successful universities like Arizona State. They’re really something to see. They take place inside football stadiums – fields of graduates in gowns and caps, all in rows like purple crops, like sprouting broccoli, stretching back as far as the eye can see. The graduation ceremonies culminate in laser light shows and fireworks and the pumping power-pop of Ellie Goulding blasting out: “Anything could happen! Anything could happen!”

Arizona State is currently Washington Monthly’s 28th best college in America and Playboy’s No4 party school.

Places like that, Albert says, “are all about getting good grades and getting a degree and becoming well integrated into the American economy.” He pauses. “I can’t completely thumb my nose at people valuing college as a good investment. People are worried about the economy, and here we are studying ancient sources and medieval sources and we don’t use textbooks and we don’t have football.”

But, he says, if that’s the only way you look at universities, you’re missing something critical.

As my son turns 16, I’ve noticed that he and his friends are wondering something that would never have crossed my mind at his age: is it even worth going to college? What’s the point? Almost half of 2013 college graduates are unemployed, or underemployed. The brilliant Andrew Rossi documentary Ivory Tower unpicks why tuition fees have increased 1,120% since 1978 – which is more than any other good or service in the entire US economy. (By comparison, food costs have gone up 200%, healthcare 600%.)

The problem stems, the film argues, from the fact that colleges are driven by the pursuit of prestige. And the way to achieve prestige is to get high on the best colleges lists. Which means, as Richard Arum, a professor of sociology and education at New York University puts it, colleges are now locked in an “arms race” to provide increasingly awesome facilities – football stadiums and luxury student condos and swimming pools with tanning ledges, like the one recently built at the University of Missouri.

Ivory Tower depicts a nightmare world in which teachers are more interested in research than in teaching, students are more interested in partying, and as a result you’ll see on sites like Rate My Professor teachers scoring through the roof because their classes are so ‘easy’. Everyone is happy. Except students are getting hundreds of thousands of dollars into debt and it’s not going on education. It’s going on tanning ledges.

“I think in the long run,” Albert Fernandez says to me, “that kind of university is probably doomed.”

This might sound crazy given that we’re sitting in a sparse room inside America’s ‘worst college’ and Albert is describing places currently bursting with money and facilities and accolades. But I understand what he means.

“Colleges like that have classes of 70, 80 students,” he says. “There’s no opportunity at all for questions. It’s simply a lecture. The professor may have one or two hours of office a week. If you’re just going to have someone lecture at you, well, you can get that from Google.”

The next day, as I watch You Tube videos of mass graduation ceremonies, I see a shot of one student, sitting among hundreds of others. He’s written on his graduation cap, ‘Thanks Wikipedia.’

Kara tells me she’d never have dreamed of picking up books like Kohlberg in real life, ‘and even if you continue to hate them, you now have read them. That sort of experience is really valuable, especially in a world where one’s own experience is so customize-able.’ She says the classroom conversations here sometimes ‘trail out into the yard, apartments, parties, into life generally. Very, very, stimulating to be around. It’s really crucial and makes you a better scholar, human being, plumber, whatever.’

Albert says he imagines some future time where places like Shimer, “or something highly specialized, will be the only type of education worth paying for.”

I hope it isn’t wishful thinking. He could be right. If Shimer survives, it could be the future.