In order to apply knowledge assessment methodologies we examine online Fitbit users communities’ discourses focusing on the quality of knowledge produced and shared about wearable sensors in that context. The online users’ community forum has a large number of entries around many topics, grouped by theme. At the same time and separately from Fitbit, the Quantified Self movement has a similar online forum where self-trackers gather as an online community discussing various topics in a similar way to Fitbit users. Conducting our research in both forums allowed us to dissociate from a particular gadget and focus more on the self-tracking aspects, as the QS movement is not associated with any particular brand. As we examined the users’ posts in order to capture elements for knowledge quality assessment, a progressive focus was used to choose themes that could best exemplify the knowledge assessment application.

As introduced earlier, in our study we focused on the analysis of pedigree of qualitative information. A number of quality categories were seen as appropriate for the type of material available from the user community forum. Having the quality categories in mind we looked for framings, factual or imagined argumentation, justifications, motivations, suggestions, appeals, assumptions and other narrative elements in the stories shared in those posts.

Two main quality categories were found to be appropriate for evaluating information pedigree in community forum: fitness for purpose and reliability. Table 1 summarises categories and subcategories.

Table 1 Analysed categories and subcategories Full size table

We looked at these forums from the perspective of digital ethnography (Domínguez et al. 2007), by observing and examining participants’ discussions in these places. The most important advantages of the unobtrusive observation via the Internet and other traditional methods such as interviews, focus groups or experimental research are the richness of the collected data and the frankness of participants which is more difficult to obtain in face-to-face conversations (Hine 2011). We have used explorative quantitative and qualitative analysis of the content posted first on the Fitbit forum and later on the Quantified Self forum. This analysis is followed by in-depth qualitative analysis with knowledge assessment methodologies described earlier.

Mapping the Topics

In this section we present the most commented topics on the forums we have analysed, grouped into several categories for further analysis.

Fitbit Forum

Fitbit offers its users access to a user community forum where people can exchange their experiences and ask for advice from other users. The forum is structured in six groups: Announcements, Big Losers, Food suggestions, Feature suggestions, General, and Help and Support. The ‘Big Losers’ category is for discussions of users that are trying to lose or have lost 75 pounds/34 kilos or more. In the ‘Food suggestions’ and ‘Feature suggestions’ categories, users are asked to provide input for the food database of Fitbit and for the development of features of the Fitbit device. By mapping the topics discussed in all the different groups, taking into account all the topics that were commented on a time period of 10 weeks (30.4.2013 to 9.7.2013), we could identify 15 categories of topics. In total 374 topics were posted in the named time period (Table 2).

Table 2 Fitbit forum Full size table

Looking at the number of topics in each category, we can see that the main concern of Fitbit users who post in the forum is to find friends and build communities. They seem to be looking for other Fitbit users in a nearby geographical location (‘Hello from Seattle WA!’), users with the same goals about weight loss and other common issues (for example ‘Group Wanted/Woman, Late 40s/100 to lose’ or ‘Guys talking about Guy Stuff and Getting Fit’) or just search friends in general who can support them by losing weight together. Another large group of topics is centred on advice on weight loss. The Fitbit users ask their fellow users’ advice on dieting, exercise and how to stay motivated (‘Need a person to teach me what I’m doing wrong to get this weight off’, ‘What to eat before work out’, ‘Frustrated and want some advice’). Many people also use the community forum to communicate their personal experience, without directly asking for advice or support. Mostly these posts are related to successful weight loss or increased fitness (for example: ‘I have lost over 100 lbs!’, ‘Just hit my first goal’). The suggestions for improvement of Fitbit are mostly focused on design, the development of the food database (where users can log their food consumption in order to calculate the daily calorie intake), and technical features. The latter concerns, for example the possibility to synchronise Fitbit with smart phones and other tracking devices and to include additional features like bar code scanning to make food monitoring easier.

Quantified Self forum

The Quantified Self forum is structured in 16 topics, with the greatest number of posts in the following threads: Apps & Tools, QS Open Forum, Sleep, Learning and Cognition. We mapped the topics discussed in different threads by taking into account all the topics commented on the period of 10 weeks (27.1.2014 to 6.4.2014). We have identified 10 categories of topics. In total 124 topics were posted in the named time period (Table 3).

Table 3 Quantified Self forum Full size table

As we can see from Table 3, the majority of posts are connected to asking for advice and sharing experiences with others. There are also a considerable number of calls to participate in research, launched by independent research institutions or companies, but this remains out of our scope. The QS forum is used less than the Fitbit to find friends, but more to exchange ideas on specific topics. Instead of losing weight together, the main issue is how to use or further develop self-tracking tools, the self-experiments, sharing experience with others and asking for advice on how to proceed with the experiment/self-tracking that they are conducting. Instead of adding friends in a social network as in the Fitbit community, the QS movement organises official meetings in locations where there are enough people involved in the movement. An interesting feature of QS forums is not the number of posts, but rather the large number of associated views and comments. For instance, although the ‘Zeo shutting down: export your data’-thread contains 305 replies, it has at the same time 59,027 views. Similarly the thread ‘Try my latest sleep hack’ has only one comment but 13,336 views.

Quality in the Quantified ‘Self’

In this section we provide the main types of quality issues that arise from the analysis of both the Fitbit and QS forums.

Fitbit

In our analysis we have focused on forum entries and threads that included some form of suggestion for the improvement of the wearable sensor technology, both in functional and usage terms, as well for the improvement of the knowledge base associated with the use of Fitbit and similar technologies. We exemplify below the types of thread contribution strategies used by participants in the threads we chose to analyse in more depth: ‘health at any size’, ‘non-wireless option’, ‘smoking log’, ‘1,750 C/day deficit…’.

In some threads forum members make connections between Fitbit technology and wider issues of health, thus initiating wider discussions about the place of wearable sensors in healthy lifestyle and health in general. Through this activity, users are interrogating different meanings of this technology. In the examined threads, users extend and connect the thread topics through posts with titles such as “a health program developed by a doctor”, “the creator of HAES [a health program] did a head-to-head study” and so on. And in a response to these posts, a wider connection to health is further discussed, as in the following post:

I had never heard of HAES until this post, but based on your explanation in a previous post, I definitely support that philosophy in my life. (…) I have more energy and feel better […] since I have started logging my food (early July), I have a much healthier relationship with food.—‘Health at any size’ thread of Fitbit community—response to User 1HAES.

As the previous example shows, in this post the user connects the use of logging food consumption and the measuring of weight as an indicator of health. Importantly, these connections are not documented with references to scientific or other authoritative material, but only based on personal experience.

In many entries to the richer debates, we can see competent observations and motivated suggestions, sometimes using specialised knowledge, derived from the participants’ professional walks of life, who intervene with mixed ‘hats’, sometimes attempting to legitimise their entries through the professional hat—replying as a pharmacist, for example. The quotes below illustrate this observation where users make suggestions by reasoning in terms of health, legal, and practical terms, as the performance of authority is enacted:

I’d like to see a wireless-free option. Why? My job often takes me into places where no wireless devices are allowed. Sometimes it’s enough to be able to turn off wireless, but in most, wireless devices are completely forbidden.—Non-Wireless thread of Fitbit community — User 1NW.

There are many such quotes where the user attempts to legitimise the opinion through professional credentials. They present themselves as medical doctors, Ph.D. students, friends of medical school students, pharmacists and so on. Also, they try to discredit people who use a device (or a measurement) the “wrong way” by stating that they are “laymen”.

On the other hand, some observations in need further justification or documentation, exhibit only the ‘I like’ corroboration. For example, there are strong claims without any specific documentation or attribution to any legitimate source. They often start with “all scientific studies…”, “physics provides a guarantee…”, “there is virtually no chance whatsoever…”, “most guidelines say…”.

In fact, many assertions correspond to auto-ethnographies including experienced facts or personal or common popular views. For example, on the “smoking log” thread a suggestion for functionality that monitors quitting smoking is made and many forum users have responded with personal experience to what they see as an improvement: “I, like many others have joined the Fitbit community in an effort to improve my overall health”, “seeing and tracking this activity should help make us confront the unhealthy habit”, “what I found the most helpful “, “I think that the standard basal metabolic rate formula just doesn’t seem to work for me. I guess I have experimental evidence (…)When I started logging my food on this site I noticed…”

In some cases further information or references are given, including rectifying information given through referencing or adding further information such as: “She also ran one of the really important clinical trials in the field…” or “Read Why We Get Fat and What We Can to About it by Gary Taubes…”.

In other cases those who follow the thread ask for evidence in the form of further information or references, after having expressed doubt about the plausibility of the assertions. Hence, the strategies for quality assurance of what is discussed and offered sometimes follow the traditional schemes of quality control in science. For example, regarding the “health at any size” thread, a user requests specifically peer-reviewed publications, considering other sources irrelevant and illegitimate:

I’m also sceptical of the above claims about the HAES vs. other diets study. It would be helpful if someone could point me to a (peer-reviewed) publication on the study and not a media report.—‘Health at any size’ thread of Fitbit community — response to User 1HAES, introducing himself as a PhD student.

Quantified Self

The QS forum is somewhat different to the Fitbit forum. The thread with the most comments is ‘Apps & Tools’. It is used by QS members, but also by developers that want to get feedback on new apps. Some of the QS forum members are not only users but also participate in creating/designing apps or devices that can help self-tracking (as well as the platforms, as we have seen in the Body Track project). This is an interesting difference from the Fitbit forum, as users turn into producers, while incorporating their own personal experience in the creation of an app. Despite the unpaid work in which users are enrolled, the inclusion of users in testing and contributing to the production of devices can be seen as very positive because they lead to a greater public engagement. These posts usually start in a similar way to this one: “A […] neuroscientist with a focus on sleep approached me to help him make an app […] I’d be really happy to get some feedback from QSers.”

Or, the app creators post a direct message to users such as:

We are going to conduct a formal study with it — so let me know also if you’d be interested in testing it out […]’Test out a new app for sleep improvement thread of QS community User dgartenberg.

Some entries ask for further information, reference or clarification to what was previously posted. The forum members want to assess the knowledge that appears in other members’ quotes by looking for themselves at the background information and by searching for the pedigree of information:

Do you have more background information that you can post here? Portland’s QS group is planning to do a meet up focused on sleep, might be worth checking out? Test out a new app for sleep improvement thread of QS community —response to User EJain.

In some cases further references are given. Sometimes they rely on self-testing or they refer to relevant medical literature or studies conducted in the same field. Or they just give examples such as: “More information can be found at the website:[…]”; “Here is info on the science behind it…”; “These findings were published in…”.

Discussions where people use auto-ethnography are also very common. By talking about their health problems and self-experimentations, users try to solve the medical problems they have by different (un) verified methods and often ask for the help of others based on their experience and consequently share their own problems and experiences. After listening to the experience of others, they make a conclusion about their own conditions:

Every morning, I wake up with one nostril almost completely congested, and the other free. Is this normal? (…) I went to four doctors about ….Of course, this is highly dubious…, I would like to ask for help from my fellow quantified selfers…—Poll: how often do you wake up with a nostril congested? Thread of QS community User Mike.

After having received answers from different users, the user who initiated the thread concludes that he has a rare condition and therefore medical doctors cannot help him. He sees a solution in self-experimentation followed by quantification.

‘Thanks everyone for their replies[…] It seems like I have a rare-ish condition.…. I’ll have to quantify how that works. …’Poll: how often do you wake up with a nostril congested? Thread of QS community —response to User Mike.

There are also many examples of auto-ethnographic studies that people conduct in order to prove/disprove an accepted truth or someone’s assumption. This quote illustrates the emerging idea of Do-It-Yourself medicine, where the accent is on our own body and a personalised approach with the goal of finding a proper cure based on experiments that is hoped to show what works the best.