Humanity has been promised salvation many, many times. We’ve been told that the key to our salvation is Christianity, Communism, Trickle Down Economics and an endless variety of quack medicines and get-rich-quick schemes. For the last decade or so, we’ve been hearing all about how Big Data is going to solve all our problems for us. It’s going to fix medicine, education and terrorism. Yay!!!

And yet…2016. How’s that Big Data working out for us? Not so hot, huh? Seems like cross-cultural relations still require people from different tribes leaving their echo chambers and taking the other person’s perspective. Seems like if you want to be healthy you still have to manage what you eat and how you exercise. Seems like education still comes down to the daily choices you make.

Ugh!!! Personal responsibility is still a thing. I thought Big Data was gonna fix everything!

Giant server farms and massive algorithms are cool and they can help but sadly the age-old challenges of the human experience are still there.

Sometimes, we want to eat our feelings and, yet, annoyingly, our choices in the moment have long-term consequences. Mark Zuckerberg and the data-driven wizards of Silicon Valley can’t manage your feelings for you. You’re stuck with them. So, you might as well use them to your advantage.

It’s an amazing thing but animals in the wild don’t need shrinks. They learn without tutors. And while they suffer a high rate of lion-induced mortality, they manage to avoid many of the diseases that plague modern humans like depression and obesity. Listen, I’m glad to live in a city and be drinking an almond milk, kinako matcha latté with great wifi. However, in the last decade and a bit, I’ve become realistic that we moderns don’t have it all figured out. In fact, there’s a lot we can learn from Papua New Guineans who run around the jungle using stone tools. In fact, I think the way these Papua New Guinean cannibals use evolution’s Small Data has more to teach us about how to solve our problems than Silicon Valley’s Big Data.

It’s worth noting that pre-modern people like the Papua New Guineans don’t have doctors. If you break your leg in the jungle, the local medicine man can’t do anything for you. You might just die. In the wet and humid tropics, simple thorn scratches and insect bites can become infected easily. Without antibiotics, these little scratches can quickly turn into a death sentence. And so, because Papua New Guineans are so bad at curing, their health and survival is heavily dependent on preventing injury and illness. The result is that they practice what Jared Diamond calls constructive paranoia in his book The World Until Yesterday.

For Jared Diamond, coming from the culture of the modern world, the rampant paranoia that Papua New Guineans displayed when leaving their villages seemed nuts. It was only with time that he came to realize that this paranoia was constructive. It was useful to overestimate dangers so that you avoided getting injured in the first place. Diamond realized that as an elderly man in his 70s, he needed more constructive paranoia in his life, notably in the shower. Imagine what would happen to healthcare in the West if everyone practiced constructive paranoia. What if we were all alive not just to the risks of injury but of not exercising and of eating poorly? 21% of U.S. medical expenses are obesity-related. 8.7% of U.S. healthcare costs are smoking related. While a tremendous amount of focus has been placed on Big Data in healthcare, your personal health outcomes mostly come down to your day-to-day choices. And given that the healthcare of a nation is just the sum of the choices of its individual citizens that reveals an important pattern: the solution to the problems Big Data identifies doesn’t lie in Big Data; it lies in Small Data.

Coming from the field of education, this is abundantly true. Big Data reveals the many problems of education in a country like America, but it’s a 10,000 foot view. From 10,000 feet, you can see that there’s a famine or a war or an epidemic. However, to actually solve the crisis, you have to get down on the ground. You have to get a reality on what is causing the famine, you have to understand why the war is being fought by talking to the two sides and you have to go figure out what microorganism is causing the disease and how it’s being spread. For the last thirteen years, I’ve been working with students from a wide range of socio-economic backgrounds. The patterns that emerge are the same and they’re the sort of patterns that Big Data doesn’t allow you to see.

A medical analogy is useful here because while fancy hospitals are attractive they’re often not the key to solving healthcare problems. Even if you can build a hospital and train world-class doctors, there’s a whole series of problems that won’t solve. A doctor only gets so much time with a patient. The patient has responsibilities. Patients need to wash their hands, eat right, exercise, not smoke, vaccinate their children and use constructive paranoia to prevent injury. They have to be active in their own healthcare.

Likewise, education requires students and parents to be active in the process. You need to find out your homework assignments and be crystal clear on the requirements. You have to look up words you don’t know. You have to analyze your mistakes for ways to improve. And you have to have enough faith in your own potential to believe that if you keep improving day by day that it will add up to real rewards. Education is an act of faith in yourself. If that’s not there, then you never really try. And that not really trying isn’t something Big Data allows you to see. Test scores show you what kids do and don’t know. They don’t show you WHY they don’t know it. Bill Gates and Mark Zuckerberg may sincerely want to reform education. They may have the Big Data of statistics. They may have billions of dollars. But what they don’t have is what those of us down on the ground have: lots and lots of Small Data. We see our students’ faces. We see that they feel disengaged from school. We see all the doodling in notebooks instead of note taking. We are in the educational trenches. We know things the Generals don’t know.

And no one is saying you don’t need Generals. You do. But wise Generals understand that statistics don’t capture everything. Just ask Robert McNamara:

McNamara is right. We all make mistakes. We know that. The question is what we do with them. And, right now, an awful lot of students are doing this with their mistakes.

Again and again, students throw away bad tests and bad essay grades. And, over time, Katie O’Brien and I found a simple solution: we made our students analyze their own mistakes. Ohhhh! They didn’t like that. They didn’t like it one bit. In fact, they often hated us for making them do it. But over time, they came to realize that their mistakes were customized feedback. They were Small Data that was telling them as an individual student where they needed to improve. They came to embrace their mistakes.

As McNamara observes, you can’t change human nature. In fact, it hasn’t changed that much in the 10,000 years since we stopped living like hunter-gatherers. What has changed a lot is our ability to control our environment; Small Data is about us controlling our own internal environment. It’s about better managing ourselves. In large part, that comes down to taking full advantage of the small data evolution hands us every day. Feelings tell us useful data. If you feel a burning sensation in your hand then how do you interpret that? Well, probably, you take it as a sign you should pull your hand away from the hot stove. But what if you didn’t know how to interpret or handle your feelings. What if instead, you just tried to self-medicate those feelings with ice cream?

In The Straight-A Conspiracy, Katie O’Brien and I taught students how to productively interpret their feelings. Students get confused all the time. The question is “How do they interpret confusion?” Well, it varies. Some students think being confused is a sign that they’re stupid. Some react to the emotion as unpleasant and simply run screaming from their textbook. But when they work with Katie O’Brien and myself, we teach them to embrace confusion. Confusion is a fun challenge. So, here is some confusion. Literally.

That’s from the Spanish version of The Straight-A Conspiracy. It shows the Chinese character for confusion and then has the parts of the character broken down…in Spanish. Given that this is an English language post, there’s a good chance you speak neither of these languages. If this was your homework, you’d probably be confused. You might feeeeeel like you didn’t know where to start. And you might want to just run away. And yet, those behaviors aren’t what it takes to succeed academically. Instead, students have to learn to interpret confusion in a productive manner. They have to discipline themselves to dig in and pick things apart. Problems in various subjects can be knotty. You have to untangle them. You have to pick the words and pieces apart bit by bit. In short, confusion is like unknotting a ball of string. And, in fact, that’s exactly what the Chinese character for confusion depicts. It’s a person kneeling (persona arrodillada) unknotting a ball of strong (deshaciendo un nudo) with the hands (con las manos). See. Not so hard. We could then go in and break down each word in Spanish and talk about conjugations but you see clearly that correctly identifying the Small Data of the emotion and acting productively is vital to making educational progress. Students in the developed world now get over a decade of schooling. Tiny choices add up to huge differences in outcomes. Katie O’Brien and I don’t have the billions of dollars or giant server farms of Silicon Valley’s ed reformers. We are teachers. All we can do is share understanding and empower students to become active in their educations.

And this gets to the heart of what makes Small Data so different from Big Data. Big Data tries to solve your problems for you. Small Data empowers you to be better at solving problems for yourself.

When I first met Tony Molina at the Rewire, I was overweight and out of shape. I ate my feelings. I went to work out because I was supposed to but the second I would get out of breath I would take a break. There were lots of unnecessary bathroom breaks. And I blamed my genetics.

In short, I did everything with my health that my students did with their educations and as Tony talked to me I got a serious dose of my own medicine. I was hearing everything I’d ever said to a student (including Tony’s son) echoed back with a certain alienated majesty. And it became clear to both Tony and me very quickly what people had been telling me for years: I was totally disconnected from my body. I was so focused on my intellectual life that I never really paid attention to what my body was doing and without paying attention you can’t develop self-awareness. I didn’t know how to interpret the feelings of soreness, fatigue, hunger and being out of breath that my body sent me. And so, I’d go, workout, back off and then eat loads because I’d earned it.

Today, I’m still overweight and out of shape. But I know why. I don’t know how to interpret the Small Data my body has been sending me for years that I’ve been busy ignoring.

Hunter-gatherers only have Small Data. Like animals, they effortlessly interpret the signals their body sends them. We moderns, on the other hand, need to consciously learn that awareness. And science can help. Below is a taste of the feelings and thoughts that students misinterpret in school and how they can interpret them more effectively.