Combining low-cost wireless EEG sensors with smartphones offers novel opportunities for mobile brain imaging in an everyday context. Here we present the technical details and validation of a framework for building multi-platform, portable EEG applications with real-time 3D source reconstruction. The system – Smartphone Brain Scanner – combines an off-the-shelf neuroheadset or EEG cap with a smartphone or tablet, and as such represents the first fully portable system for real-time 3D EEG imaging. We discuss the benefits and challenges, including technical limitations as well as details of real-time reconstruction of 3D images of brain activity. We present examples of brain activity captured in a simple experiment involving imagined finger tapping, which shows that the acquired signal in a relevant brain region is similar to that obtained with standard EEG lab equipment. Although the quality of the signal in a mobile solution using an off-the-shelf consumer neuroheadset is lower than the signal obtained using high-density standard EEG equipment, we propose mobile application development may offset the disadvantages and provide completely new opportunities for neuroimaging in natural settings.

Funding: This work is supported in part by the Danish Lundbeck Foundation through CIMBI - Center for Integrated Molecular Brain Imaging and a post-doctorate grant for CS, and by the H. C. Ørsted Foundation. No additional external funding was received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Introduction

In the last few years, the research communities studying human behavior have gained access to unprecedented computational and sensing power that basically “fits into a pocket”. This has happened for both specialized equipment used for building research tools, such as Reality Mining Badges [1] or accelerometer sensors [2], and for consumer-grade, off-the-shelf devices. Smartphones and tablets are capable of sensing, processing, transmitting, and presenting information. This has already had a significant impact on many research domains, such as social science [3], computer-human interaction [4], and mobile sensing [5], [6]. In neuroscience there is a widely recognized need for mobility, i.e., for devices that support quantitative measurements in natural settings [7]–[9]. Here we present our work on the Smartphone Brain Scanner, investigate the feasibility of off-the-shelf, consumer-grade equipment in a neuroscience context, and build a mobile real-time platform for stimulus delivery, data acquisition, and processing with a focus on real-time imaging of brain activity.

Consumer-grade neuroheadsets, capable of recording brain activity generated by post-synaptic potentials of firing neurons, captured through electrodes placed on the scalp using Electroencepahlography (EEG), have only recently made mobile brain monitoring feasible. Seen from a mental state decoding perspective, even a single channel EEG recording measuring the changes in electrical potentials (based on a passive dry electrode positioned at the forehead and a reference typically placed on the earlobe), allows for measuring mental concentration and drowsiness by assessing the relative distribution of frequencies in brain-wave patterns throughout the day. Simply measuring the dynamic variability of brain-wave frequency components in a mobile scenario may be translated into neural signatures, e.g., reflecting whether a user is on the phone while driving a car [10]. Similarly, positioning the single EEG electrode headband over the temple may provide the foundation for building a Brain-Computer Interface (BCI) utilizing the ability to capture steady-state visual-evoked potentials (SSVEP) from the visual cortex when looking at flashing lights patterns, and thereby design a BCI interface for prediction with high accuracy and no previous training when a disabled user is focusing on a specific area of a screen, based on the time-locked EEG traces automatically generated as multiples of the particular flashing light frequencies [11].

As an example of the underlying technology used in several consumer products, the ThinkGear module manufactured by NeuroSky (http://www.neurosky.com/Products/ThinkGearAM.aspx) integrates a single dry electrode (reference and ground) attached to a headband. Essentially a system on a chip, it provides A/D conversion and amplification of one EEG channel, capable of capturing brain-wave patterns in the 3–100 Hz frequency range, recorded at 512 Hz sampling rate. Consumer neuroheadsets, such as those manufactured by Emotiv (http://www.emotiv.com) provide low-density neuroimaging based on 16 electrodes and typically support real-time signal processing in order to complement standard EEG measures with aggregate signals, which provide additional information on changes in mental state, or facilitate control of peripheral devices related to games. Their portability and built-in wireless transmission makes them suitable for the development of fully mobile systems, allowing for running EEG experiments in natural settings. The improved comfort of these mobile solutions also allows for extending neuroimaging experiments over several hours. Furthermore, the relatively low cost of the neuroheadsets and mobile devices potentially opens new opportunities for conducting novel types of social neuroscience experiments, where multiple subjects are monitored while they interact [12], [13].

However, such ‘low-fi’ mobile systems present a number of challenges. In real-time applications requiring signal processing to be performed with the lowest possible delay in order to present feedback to the user, the limited computational power of mobile devices may be a constraint. A solution might be to offload parts of the processing to an external server and retrieve the processed results over the network. This approach, however, requires network connection, possibly with low and constant delays, as well as more complicated client-server architecture. Also in terms of battery life, the local computation is more power-efficient than continuous transmission to the server and back. Consumer-grade mobile devices also present technical challenges for writing high-quality software; the devices operate on systems that are not real-time (RTOS), as they do not guarantee certain delays in data processing, and as such are ill-suited for time-sensitive tasks. These limitations might also affect timing of visual or auditory stimuli presentation, as well as synchronization with other sensors. From a neuroscience perspective, low-resolution recordings and artifacts induced in a mobile setup both present significant challenges. Noise and confounds are introduced by movement of the subject and electrical discharges, while the positioning of the electrodes might be less than ideal when compared to a standard laboratory EEG setup [14]–[17]. Nevertheless, we hold that these drawbacks are clearly offset by the advantages of being able to conduct studies incorporating larger groups of subjects over extended periods of time in more natural settings. We suggest that mobile EEG systems can be considered from two viewpoints: as stand-alone portable low-fi neuroimaging solutions, or as an add-on for retrieving neuroimaging data under natural conditions complementary to standard neuroimaging lab environments.

In terms of software programming, creating a framework for applications in C++ rather than in prevalent environments such as MATLAB, while approaching the problem as a smartphone sensing challenge, might enable new types of contributions to neuroscience. The Human-Computer Interaction (HCI) community is already starting to apply consumer-grade headsets to extend existing paradigms [18], thus incorporating neuroscience as a means to enhance data processing. Similarly, the availability of low-cost equipment means even general ‘hacker-and-tinkerers’ audiences will almost certainly gain interest in using neuroscience tools (http://neurogadget.com). We see a great value in the emerging potential of entirely new groups of researchers and developers becoming interested in neuroscience and obtaining tools allowing them to develop new kinds of applications.

In a previous communication [19], we discussed opportunities and challenges in mobile and portable EEG. Here we address the foundations of the Smartphone Brain Scanner system, focusing on engineering and technical aspects of both software and hardware components. We describe the computational architecture of the framework, and discuss timing, reliability, and quality of the obtained signal. In particular we report on the results of a validation experiment comparing the system with a conventional EEG acquisition system in a prototypical application.