Another New Year, another list of resolutions to make… and no doubt break. For some, 2013 may begin by going to the gym, or giving up smoking. But what about a more thorough and personal approach towards self-improvement, like monitoring your thoughts, e-mails, or even your bowel movements? If the answer is yes, then perhaps this is the year that you’ll be joining a growing group of people who are using technology to measure their lives in extraordinary detail.

Probably since the dawn of humanity, people have been fascinated by even the most minute details of their lives, and kept track of what was going on in their bodies and minds. The Roman philosopher Seneca tracked the food he ate and what he dreamt at night. Benjamin Franklin consistently recorded his performance on 13 measures, such as cleanliness, frugality and overindulgence, believing it would keep him virtuous. Engineer and architect Buckminster Fuller nicknamed himself “guinea pig b” and kept a diary on his daily life and ideas.

But in the past, record-keeping was time-consuming, requiring a commitment that only a very few had the patience to muster. Others cared, but not THAT much. Today, that’s changing, as it becomes easier to track everything, from diet to mood to sleep quality. Smartphones come equipped with features like GPS, accelerometers and gyroscopes that can record your activity, location and other vital statistics. Millions of fitness-focused or diet-conscious people track their workouts or their desserts with apps and devices like Fitbit that track physical activity or calories burned. New technology also makes it easier to share results with others – who doesn’t want to brag the first time they run more than 5 miles (8km), or be praised when they’ve managed to lose 10 pounds (5kg)?

A growing band of devotees centered in the United States, called the Quantified Self movement, has taken the effort to another level. Its members track a whole list of measures and push the technology available to keep tabs on their ideas, their mental health, even their microbiome – the colonies of bacteria that live on or inside their bodies. Some might scoff at their efforts to keep spreadsheets documenting their mood, diet or sleep patterns. “But the insights that we could learn from having all this quantified self data available are almost unfathomable,” says Amy Robinson, an organiser of Quantified Self Boston, a group that includes about 650 members and holds bimonthly meet-ups to share ideas and progress.

“Quantified self helps you keep a reference to how you lived not only the day before but two weeks before or a year before,” Robinson says. “You know when you achieve your best, you know when you exceed your best and it makes that possible by simply having the numbers there.”

Yet such lofty personal goals come down to something that couldn’t be more mundane: manually recording the number of daily bowel movements or every bite of food consumed.

Lawrence David spent a year tracking 300 measures about his health and behaviour – including the microbes in his digestive system – while he was a graduate student at the Massachusetts Institute of Technology. David created a self-tracking system by tinkering with an iPhone app designed to record and manage business cards. Every meal became a new “card” entry, with hundreds he had to type in by hand over the year – one night’s entry could be cheeseburger and fries, the next pepperoni pizza. “One of the few datasets out there is going to be full of junk food,” he says, confessing to a personal weakness.

He got to the point where he hated the process so much that he didn’t want to eat because then he’d have to record it. “It was terrible,” he admits. “Super tedious.” But when his boss, associate professor Eric Alm, got food poisoning while self-tracking, it was the first time the microbial pattern of human diarrhoea had been followed in real-time. Sounds gross, but understanding the progress of food poisoning in such a manner could be an important step toward developing treatments for it.

And other than confirming his love of junk food, he did learn some useful things about himself. David discovered that he wasted a lot of time on e-mail during the day, and wasn’t as productive as he thought he was late at night. Now, he saves most of his e-mailing for the evenings – instead of disrupting his research to respond – and doesn’t feel guilty when he takes a night off. He’s stopped tracking most of the other measures, though.

Number power

Like David, others who use technology to self-track rely on a mix of mind-numbing and automated measures, and they talk about the greater good, as well as what they’ve learned about themselves. The Quantified Self movement originated in San Francisco, after it was first proposed by Wired editors Gary Wolf and Kevin Kelly in 2007. As of October 2012, the Quantified Self movement counted more than 70 meet-up groups worldwide with over 5,000 members – of course, they keep track of that sort of thing. (There’s actually a greater number of self-tracking apps on mobile phones than members – around 7,500 at the moment.) Many of these self-trackers are researchers, computer scientists, or simply tech-savvy, 21st-century Benjamin Franklins who think the act of collecting so much data has the power to transform themselves and the rest of society.

Meet-ups are part 7-minute show-and-tells, part social hour. At her first meet-up, in Alabama, Amy Robinson was the only one who had tried quantifying anything, while other attendees were just curious about the idea. In Boston, as you might expect from being an academic and tech hub, many of the participants have developed their own hacks and want to trade tips on how to maximise their record-keeping.

Robinson says she has long been fascinated with where ideas come from, so she started sending herself an e-mail whenever a good one popped into her head. Recently, she collected six months of these e-mails – 770 in all – analysed them, and presented the results at a Quantified Self meetup in Boston in October. Among the themes that came up most often were “science”, “beautiful” and “health.” She found that she had the most new ideas immediately after attending an inspiring conference, travelling or experiencing something new. Doing this exercise helped feed her fascination with the human mind, she told the group, as well as “discover more ways to think about myself and discover how my mind works relative to other people’s.”

Melanie Swan, founder of DIYgenomics, a California-based non-profit that promotes personalised medicine, has presented her findings at several Bay Area meet-ups, and she says meet-up attendees have changed as the movement has evolved. At first, the typical person was obsessive, “someone who is really rigorous about self-tracking either because of a health reason or they’re compelled to improve their mental performance or whatever,” she says. “But who’s now at the meet-ups is a much broader slice, though I would say it’s still a pretty tech-forward thinking kind of audience – anybody interested in improving an aspect of their health.”

Power of the nudge

While their interest in self-quantification is technological, another factor is a more general one: the growing obsession with self-improvement over the past few decades, says Natasha Schull, a cultural anthropologist at the Massachusetts Institute of Technology.

Weight Watchers has been telling people since the 1970s that if they want to lose weight, they need to track what they eat. The field of behavioural economics, which studies the connection between thoughts and financial decisions, has found that if you alter peoples’ social environment, they might be nudged into conducting more rational behaviour. For instance, if school cafeterias display fruit before fried chicken, students are more likely to eat fruit because they grab the first food they see. And having people track their actions makes a difference, says Schull, encouraging more people to make the effort to track.

Self-tracking might not have been such a fringe activity in the past, either. There were plenty of people keeping diaries and self-accounts in the 1800s and 1900s, says Schull. “Even having a watch and being mindful of time is a form of self-regulation that went along with industrialisation,” she says.

As long as there have been people tracking themselves, there have been people mocking those trackers, Schull adds. D.H. Lawrence wrote a piece making fun of Franklin’s self-obsession and the cockiness embedded in his 13 virtues, including this description: “He was a little model, was Benjamin. Doctor Franklin. Snuff-coloured little man! Immortal soul and all!”

Behind such insults lies the idea that tracking the minutiae of one’s life can be pointless, an end in itself. Others worry that it could be used for self-destructive purposes, like an anorexic tracking how little she eats. “It seems to me in the contemporary US example, the means and the end are blurring,” says Schull, who co-taught a class at Harvard University in the autumn called “Self as Data”. “I think people are still motivated – at least they say they are – by some kind of transformation, but it’s seen as something that doesn’t really end.” Like the Alcoholics Anonymous model, where people who were once alcoholics are always recovering, Schull says. “It’s a different model of what health is.”

Weight and see

For Mark Gerstein, a professor of bioinformatics at Yale University, tracking his weight helped him keep it under control. “If you watch things, you tend to do better, by virtue of being aware of them,” says Gerstein, who has a wifi-enabled bathroom scale that automatically records his weight and feeds it into a database.

Gerstein has also been tracking his asthma symptoms in hope of being able to reduce his medication. He says he sees a little correlation between his exercise and his peak air flow, and thinks the more he can measure about himself, the more he’ll know about what triggers his wheezing and the better he’ll be at controlling it. It’s not just Gerstein’s data that’s important, this kind of information from thousands of people who have asthma could become much more powerful when it’s aggregated. “There’s a whole other level of what happens when all this information flows together and you can look for patterns and trends,” he says.

But not all self-quantification efforts are successful. Schull’s student, Sneha Khullar, began – reluctantly – tracking several measures as a class assignment. In her final presentation to the class in December, she explained that she enjoyed using a pedometer to track her footsteps and an app called RescueTime, which showed she is most productive on Fridays – now that she’s given up partying. But writing down everything she ate was too much of a drag, and piling up chocolate wrappers didn’t help her moderate her sweet tooth.

According to Khullar, a student at the Harvard Graduate School of Design, self-tracking needs to be easy enough to carry out routinely, but not so easy that it can be ignored. Having to convert miles into kilometres so she could figure out how much she had run required just enough effort to get her invested in the process, with an incentive to keep running more. “I could not have imagined I could run more than 5 miles,” she proudly told her classmates. But, these tools were way more successful in behaviour-enhancing than in controlling bad habits, she admitted.

And perhaps this barrier to participation is key to whether self-tracking will fully make the leap from tech obsessives to the mainstream public. “It’s a question of when does it become easy and when does it become useful,” says Stephen Wolfram, creator of the technical computing software Mathematica. He says he’s been self-tracking for a quarter of a century – everything from his personal health to his e-mail to the use patterns of every room in his house. But there are some things even he declines to measure. He doesn’t bother with skin conductivity measures, for instance, because he doesn’t like wearing the devices. He doesn’t wear a camera around his neck, “because it’s not socially realistic for the life I lead – maybe if I were much younger.” And he doesn’t record his side of phone conversations because speech recognition software still isn’t good enough to provide an accurate transcript. Why would he want a transcript of every conversation? “Am I repeating myself? You can immediately check,” he says.

Privacy matters

The next crucial step for self-tracking, several people said, will be the ability to integrate different data streams. Right now, the wifi-enabled scale doesn’t talk to the program that monitors blood pressure, diet or sleep – but such integration will come eventually. Maybe someday it will even link up with “smart” toilets that can read shifting microbial populations and warn of impending illness. The key will be finding ways to make the data meaningful, rather than just overwhelming. And to keep it private.

Yale’s Gerstein says he worries a lot about the privacy aspects of all thins data. Right now, it seems harmless to share information about our sleep patterns, genes and microbial populations. But one day, that information could be more meaningful – and could be used against us. Sleep patterns could be relevant, say, in a court case about a traffic accident. People who post genetic information now are revealing things about their future children and grandchildren. “I think the fear is people will pile up a lot of information about themselves and share it without fully seeing the implications,” Gerstein says.

Robinson says there’s no question the Quantified Self movement she is part of will have to address privacy issues. “Let the people who want to share their data in a way that maintains their privacy, but also helps other people,” she says.

And doctors are going to need to learn how to deal with all the data their patients are beginning to bring them. Right now, Robinson says, people in the Quantified Self movement “want to share their information with their doctor, but their doctor doesn’t really know what to do with it.”

Wolfram, the founder and CEO of Wolfram Research and founder of the technology company Wolfram|Alpha, thinks the data will eventually redefine what doctors do. A doctor would be hard-pressed to make sense of the data once people are wired to track their heart rate, blood sugar levels, blood pressure and dozens of other measures. But computers will be able to easily spot patterns and trends, diagnosing far more accurately than doctors can today, and far earlier in the disease process, says Wolfram, who is developing analytics to meet this need. He thinks that eventually we’ll go from sensor-based data to being able to treat whatever concern the technology diagnoses.

If so, perhaps a Quantified Selfer will figure out a way to measure whether this actually improves our health.

If you would like to comment on this story or anything else you have seen on Future, head over to our Facebook page or message us on Twitter.