Research: Projects
The mhealth Research Group explores how mobile and ubiquitous sensing might be extended and then used to support health and wellness through personal health informatics. As such, some projects have more of a technical, or computer-science flavor, and other projects have more of a public health or behavioral-science flavor. Most projects, though, combine both areas of interest.
Listed here are our active projects, projects that are current dormant but that we would like to restart given student interest, and projects that are completed that are unlikely to be restarted (although all of our past projects are influencing what we do in the future). Nearly all of our projects involve an interdisciplinary, team-science approach.
Projects generally touch upon one or more of these overlapping areas of interest:
Listed here are our active projects, projects that are current dormant but that we would like to restart given student interest, and projects that are completed that are unlikely to be restarted (although all of our past projects are influencing what we do in the future). Nearly all of our projects involve an interdisciplinary, team-science approach.
Projects generally touch upon one or more of these overlapping areas of interest:
- Activity and context recognition using data from smartphones, smartwatches, other wearable sensors, and/or ubiquitous computing devices
- Human-in-the-loop behavior and habit recognition with limited training data
- Computational modeling of behaviors and behavioral theory, especially to support behavior change and behavior maintenance
- Extensions to context-sensitive ecological momentary assessment (a.k.a. experience sampling), such as microinteraction ecological momentary assessment (uEMA)
- Just-in-time adaptive interventions (JITAIs) that use real-time sensing
- Measuring physical activity, sedentary behavior, and/or sleep (at population scale), as well as other health-related behaviors and contexts
- Using mobile and ubiquitous computing to support active transportation (via bicycling)
- Accelerating research on the topics above through generation of shared datasets, software tools, and living laboratories
- Persuasive technology
Active or planned projects
Development of Algorithms for Detecting the Activities of Adults and Children from Wearable Sensors
|
We are working on a variety of projects studying how to use mobile sensor data, especially from accelerometers, to detect physical activity (type, duration, and intensity), sedentary behavior, and sleep in adults and children.
Collaborator(s): EveryFit, Inc., Stanford Medical School, Case Western Reserve University |
Accelerometer data from the wrist and ankle, and manually-labeled activities. We are developing algorithms to detect such everyday behaviors.
|
Developing a Computational Pathway and Algorithms to Summarize Large-Scale Accelerometer Data
|
We are developing easy-to-use methods and algorithms for summarizing large amounts of accelerometer data collected in important health surveillance studies such as the NHANES and the UK Biobank. Our goal is to advance research by creating non-proprietary and open-source tools and methods that allow large-scale data data-driven discovery.
Collaborator(s): Dinesh John (Northeastern University) |
We are creating a simple way for researchers to run algorithms on 18,000+ raw accelerometer datasets from the NHANES study.
|
Developing End-user-driven Interactive and Real-Time Activity Recognition Algorithms
|
This is an ongoing effort to develop practical activity recognition systems for smartphones using wearable sensing. Doing so will require appropriate human-in-the-loop activity training and algorithm refinement, and the use of common sense information by the algorithms themselves. Our goal is to create practical systems that can be used for real-time measurement of behavior to support novel just-in-time mobile health systems.
|
|
Using Temporally-Dense Microinteractions to Measure and Promote Behavior Change
|
The purpose of this project is to study how “microinteractions” – quick interactions with smartwatches, or even devices like Google Glass – can be used to create new ways to continuously measure health behavior and provide opportunities for just-in-time health intervention delivery to support health behavior change and behavior maintenance.
Collaborator(s): Genevieve Dunton (USC) |
In micro-EMA on a smartwatch, only single questions with a simple answer are asked, but at a high temporal density.
|
Changing Commuting Culture: Virtual Bicycle Highways
|
This project explores how mobile technology can be used to support active commuting via bicycling, where individuals who would normally be commuting by car may instead commute via bicycle. They may do this because of a novel technological system that would create a car-free riding experience, dramatically enhancing comfort and safety. We are looking for students interested in bicycling who want to help with this project, which is in the early stages. We are building a state-of-the-art bicycle simulator, to help support future research in this area.
|
Concept image showing one aspect of the Virtual Bicycle Highways concept.
|
Interactive Visualization Tools for Type 1 Diabetes Treatment Decision Management
Type 1 diabetes (T1D), also called insulin dependent diabetes and juvenile diabetes, is an autoimmune disease that afflicts 1.25 million Americans. There are 40 thousand new diagnoses each year, and almost half of those are children and adolescents under age 20 years. The costs for our health care system are enormous, estimated at $14 billion annually. T1D is incurable, and people with T1D are estimated to lose over 10 years from their life expectancy. We are designing, building, and evaluating interactive visualization tools to help T1D patients and their caregivers make treatment decisions. In these visualizations we are showing data we collect using multiple devices and have data on the patient blood glucose levels, insulin administered, food eaten, exercise, and stress to name a few. Our tools are used by patients to understand the trends between scheduled events such as mealtimes, bedtimes, and overnight as well as irregular events like periods of exercise, stress, and illness. With this information, patients will be able to make more informed changes to their treatment protocols.
Collaborators: Cody Dunne (Northeastern), Boston Children’s Hospital
Lifestyle Modification and Social Support via a Text-Messaging Digital Companion
The goal of this project is to develop a completely new way of supporting lifestyle modification using sensor-enabled, just-in-time and context-sensitive text messaging, with a combination of automated feedback and expert feedback, but in a scalable way.
Crowd-Sourced Annotation of Longitudinal Sensor Data to Enhance Data-Driven Precision Medicine for Behavioral Health
|
Longitudinal sensor data collected passively from wearable activity monitors and mobile phones will transform behavioral science by allowing researchers to use “big data,” but at the person-level, to understand how behavior and related environmental exposures impact health outcomes and personalize health intervention and research. We propose to develop and test a system that permits typical mobile application game players to help scientists improve this type of data, by adding additional annotations that enrich the data, making it more useful for behavioral science and more amenable to automatic processing. This will help researchers to better understand how individual-level behaviors relate to health outcomes in current research studies that collect personal-level sensor data such as NHANES and the Women’s Health Study, and future big data ventures such as the new Precision Medicine Initiative.
Collaborators: Dinesh John (Northeastern University), Seth Cooper (Northeastern University) |
Prototype of a game designed to help researchers, by having game players label activities using accelerometer data.
|
Novel Statistical Models for EMA Studies of Physical Activity
|
This study is focused on the development and testing of novel multilevel statistical methods to examine the effects of subject-level parameters (variance and slope) of time-varying variables in ecological momentary assessment (EMA) studies of physical activity. The results will have the potential to make notable methodological and substantive contributions for analysis of EMA data and in understanding the complex effects of contexts and psychological states on physical and sedentary activity. Given the detrimental health effects and enormous costs of physical inactivity, such information could be of significant value to efforts to improve public health.
Collaborators: University of Southern California, University of Chicago |
Part of the interface of the software developed for researchers interested in new multi-level statistical methods.
|
Just-in-time adaptive feedback systems to assist individuals with spinal cord injuries
|
We have developed and validated a smartphone-based Physical Activity Monitoring System (PAMS) that uses machine learning algorithms to detect various wheelchair-based physical activities (PAs) in community settings, estimate energy expenditure, and provide real-time feedback about PA levels to individuals with spinal cord injury. The objective of this study is to develop algorithms that incorporate each individual’s automatically detected PA level and a clinician’s PA level recommendation to provide a smartphone application that helps a person set safe and highly personalized physical activity goals. By adapting the goals in real-time based on the person's actual behavior, the system aims to keep the individual feeling positive and motivated towards a change in the PA behavior.
Collaborators: Temple University |
In the PHIRE project, a smartphone, smartwatch, and wheel sensor are used to estimate energy expenditure.
|
Measuring the impact of a community bikeway
|
As part of our effort to explore the Virtual Bicycle Highway concept, we are gathering data on the Minuteman Commuter Bikeway, which runs from Bedford, Massachusetts to Arlington, Massachusetts. This is a great example of a bikeway that runs from the suburbs to the city. We are exploring how many people have changed their active commuting patterns because of the bikeway, and how much the bikeway has impacted the surrounding community.
|
The Minuteman Bikeway runs through four towns and is heavily used by commuters. But there is more to learn about how and why it is used.
|
Context-Sensitive Experience Sampling
|
We are studying how real-time activity and context recognition on mobile phones can be used for context-sensitive experience sampling (or context-sensitive ecological momentary assessment). Such systems could improve the quality of self-report data while minimizing burden on scientific participants, but at the same time leading to higher-quality data gathering. In addition, the systems may enable new forms of computer-assisted end-of-day recall, also valuable for certain types of research studies.
Collaborators: Genevieve Dunton (USC) |
Partial screenshot of the MobileTeens application, that used sensor data to help teens label their everyday activities using context-sensitive experience sampling.
|
Development of Algorithms for Detecting the Activities of Adults and Children from Wearable Sensors
|
We are working on a variety of projects studying how to use mobile sensor data, especially from accelerometers, to detect physical activity (type, duration, and intensity), sedentary behavior, and sleep in adults and children.
Collaborator(s): EveryFit, Inc., Stanford Medical School, Case Western Reserve University |
Accelerometer data from the wrist and ankle, and manually-labeled activities. We are developing algorithms to detect such everyday behaviors.
|
Developing a Computational Pathway and Algorithms to Summarize Large-Scale Accelerometer Data
|
We are developing easy-to-use methods and algorithms for summarizing large amounts of accelerometer data collected in important health surveillance studies such as the NHANES and the UK Biobank. Our goal is to advance research by creating non-proprietary and open-source tools and methods that allow large-scale data data-driven discovery.
Collaborator(s): Dinesh John (Northeastern University) |
We are creating a simple way for researchers to run algorithms on 18,000+ raw accelerometer datasets from the NHANES study.
|
Developing End-user-driven Interactive and Real-Time Activity Recognition Algorithms
|
This is an ongoing effort to develop practical activity recognition systems for smartphones using wearable sensing. Doing so will require appropriate human-in-the-loop activity training and algorithm refinement, and the use of common sense information by the algorithms themselves. Our goal is to create practical systems that can be used for real-time measurement of behavior to support novel just-in-time mobile health systems.
|
|
Using Temporally-Dense Microinteractions to Measure and Promote Behavior Change
|
The purpose of this project is to study how “microinteractions” – quick interactions with smartwatches, or even devices like Google Glass – can be used to create new ways to continuously measure health behavior and provide opportunities for just-in-time health intervention delivery to support health behavior change and behavior maintenance.
Collaborator(s): Genevieve Dunton (USC) |
In micro-EMA on a smartwatch, only single questions with a simple answer are asked, but at a high temporal density.
|
Changing Commuting Culture: Virtual Bicycle Highways
|
This project explores how mobile technology can be used to support active commuting via bicycling, where individuals who would normally be commuting by car may instead commute via bicycle. They may do this because of a novel technological system that would create a car-free riding experience, dramatically enhancing comfort and safety. We are looking for students interested in bicycling who want to help with this project, which is in the early stages. We are building a state-of-the-art bicycle simulator, to help support future research in this area.
|
Concept image showing one aspect of the Virtual Bicycle Highways concept.
|
Interactive Visualization Tools for Type 1 Diabetes Treatment Decision Management
Type 1 diabetes (T1D), also called insulin dependent diabetes and juvenile diabetes, is an autoimmune disease that afflicts 1.25 million Americans. There are 40 thousand new diagnoses each year, and almost half of those are children and adolescents under age 20 years. The costs for our health care system are enormous, estimated at $14 billion annually. T1D is incurable, and people with T1D are estimated to lose over 10 years from their life expectancy. We are designing, building, and evaluating interactive visualization tools to help T1D patients and their caregivers make treatment decisions. In these visualizations we are showing data we collect using multiple devices and have data on the patient blood glucose levels, insulin administered, food eaten, exercise, and stress to name a few. Our tools are used by patients to understand the trends between scheduled events such as mealtimes, bedtimes, and overnight as well as irregular events like periods of exercise, stress, and illness. With this information, patients will be able to make more informed changes to their treatment protocols.
Collaborators: Cody Dunne (Northeastern), Boston Children’s Hospital
Lifestyle Modification and Social Support via a Text-Messaging Digital Companion
The goal of this project is to develop a completely new way of supporting lifestyle modification using sensor-enabled, just-in-time and context-sensitive text messaging, with a combination of automated feedback and expert feedback, but in a scalable way.
Crowd-Sourced Annotation of Longitudinal Sensor Data to Enhance Data-Driven Precision Medicine for Behavioral Health
|
Longitudinal sensor data collected passively from wearable activity monitors and mobile phones will transform behavioral science by allowing researchers to use “big data,” but at the person-level, to understand how behavior and related environmental exposures impact health outcomes and personalize health intervention and research. We propose to develop and test a system that permits typical mobile application game players to help scientists improve this type of data, by adding additional annotations that enrich the data, making it more useful for behavioral science and more amenable to automatic processing. This will help researchers to better understand how individual-level behaviors relate to health outcomes in current research studies that collect personal-level sensor data such as NHANES and the Women’s Health Study, and future big data ventures such as the new Precision Medicine Initiative.
Collaborators: Dinesh John (Northeastern University), Seth Cooper (Northeastern University) |
Prototype of a game designed to help researchers, by having game players label activities using accelerometer data.
|
Novel Statistical Models for EMA Studies of Physical Activity
|
This study is focused on the development and testing of novel multilevel statistical methods to examine the effects of subject-level parameters (variance and slope) of time-varying variables in ecological momentary assessment (EMA) studies of physical activity. The results will have the potential to make notable methodological and substantive contributions for analysis of EMA data and in understanding the complex effects of contexts and psychological states on physical and sedentary activity. Given the detrimental health effects and enormous costs of physical inactivity, such information could be of significant value to efforts to improve public health.
Collaborators: University of Southern California, University of Chicago |
Part of the interface of the software developed for researchers interested in new multi-level statistical methods.
|
Just-in-time adaptive feedback systems to assist individuals with spinal cord injuries
|
We have developed and validated a smartphone-based Physical Activity Monitoring System (PAMS) that uses machine learning algorithms to detect various wheelchair-based physical activities (PAs) in community settings, estimate energy expenditure, and provide real-time feedback about PA levels to individuals with spinal cord injury. The objective of this study is to develop algorithms that incorporate each individual’s automatically detected PA level and a clinician’s PA level recommendation to provide a smartphone application that helps a person set safe and highly personalized physical activity goals. By adapting the goals in real-time based on the person's actual behavior, the system aims to keep the individual feeling positive and motivated towards a change in the PA behavior.
Collaborators: Temple University |
In the PHIRE project, a smartphone, smartwatch, and wheel sensor are used to estimate energy expenditure.
|
Measuring the impact of a community bikeway
|
As part of our effort to explore the Virtual Bicycle Highway concept, we are gathering data on the Minuteman Commuter Bikeway, which runs from Bedford, Massachusetts to Arlington, Massachusetts. This is a great example of a bikeway that runs from the suburbs to the city. We are exploring how many people have changed their active commuting patterns because of the bikeway, and how much the bikeway has impacted the surrounding community.
|
The Minuteman Bikeway runs through four towns and is heavily used by commuters. But there is more to learn about how and why it is used.
|
Context-Sensitive Experience Sampling
|
We are studying how real-time activity and context recognition on mobile phones can be used for context-sensitive experience sampling (or context-sensitive ecological momentary assessment). Such systems could improve the quality of self-report data while minimizing burden on scientific participants, but at the same time leading to higher-quality data gathering. In addition, the systems may enable new forms of computer-assisted end-of-day recall, also valuable for certain types of research studies.
Collaborators: Genevieve Dunton (USC) |
Partial screenshot of the MobileTeens application, that used sensor data to help teens label their everyday activities using context-sensitive experience sampling.
|
Projects on hiatus
These projects are currently not being developed, but we would like to restart them given appropriate student interest.
Operant Conditioning for Motivating Behavior Change
How can mobile devices motivate behavior change? One way might be to wait patiently, automatically detecting when someone does something just a bit better than what that person normally does using sensors such as accelerometers... and then providing context-sensitive, tailored encouragement. In other words, software on a mobile device might use computational sensing to detect behavior, and operant conditioning to motivate gradual behavior change.
Automatic Detection of Smoking Behavior
|
The goal of this pilot project was to develop and evaluate real-time pattern recognition algorithms that run on mobile phones and process data from miniature, wearable motion sensors (accelerometers) to detect patterns of smoking behaviors in free-living individuals.
Collaborator(s): M.D. Anderson Cancer Research Center |
|
Generating a Free, High-Quality Food Product Database Using Games with a Purpose
|
The specific aim of this pilot project was to explore the use of “games with a purpose” approach as a proof-of-concept to construct a UPC and nutrient database.
Supported by: NIH/NCI |
|
Telemetric Assessment of Movement Stereotypy in Children with ASD
A study with the Groden Center, a school for autistic children, and Prof. Matthew Goodwin at Northeastern, that has eplored the use of wireless accelerometers for automatic detection of autistic stereotypies. We would like to extend our past work to build real-time, interactive interventions for autistic children and their caregivers and teachers.
Collaborator(s): Groden Center, University of Rhode Island, Prof. Matthew Goodwin (Northeastern)
Supported by: National Alliance for Autism
Completed Projects
These projects are currently not being further developed.
Optimizing Hospital Workflow and Quality Through Patient Engagement
|
This project extended the Hospital Buddy, a computer agent that provides continual health counseling and companionship during a patient’s hospital stay via a bedside touch screen computer equipped with sensors (RFID tags on staff, accelerometer on patient, acoustics) that can detect certain events occurring in the hospital room.
Collaborator(s): Timothy Bickmore (Northeastern), Harriet Fell (Northeastern), Boston Medical Center |
A patient using the Hospital Buddy system in a hospital.
|
Enabling Population Scale Physical Activity Measurement on Mobile Phones
|
A study with Stanford School of Medicine to create novel health monitoring tools for mobile phones. This work led to the Wocket wearable accelerometer sensors, which were used for a time in several projects but have gone out of the date. The project included a supplement to develop mobile context-sensitive ecological momentary assessment software for mobile phones (“Extensible Platform for Implementing Experience Sampling on Mobile Phones”). Work continues today on this topic, although through different sources of funding and we are no longer making hardware.
Here is a flash video of the concept. There is still a need for this type of system to be produced commercially. Collaborator(s): Stanford Medical School |
Wocket sensors were thin, lightweight custom sensors we designed for capturing multi-limb accelerometer data in research studies, using common mobile phones.
|
Brain-Computer Interface for Signaling Changes in Physiological States
The major goal of this pilot project was to establish the feasibility of using fNIRS to detect major changes in emotional state and to use a mobile system for context-sensitive self-report on that state change. The prototype suffered from substantial motion artifacts that we were not able to adequately filter out.
Collaborator(s): Dana Brooks (Northeastern), Lisa Feldman Barrett (Northeastern), Matthew Goodwin (Northeastern), Deniz Erdogmus (Northeastern)
Cell Phone Intervention Trial for You (CITY)
|
This was a five-year study to develop and evaluate (in a randomized clinical trial) sensor-enabled mobile phone technology to assist young adults aged 18-35 with long-term weight loss and weight management for two years. We designed and created the mobile app and deployed it with over 200 people.
Collaborator(s): Duke Medical School |
Screenshots from the CITY Android application.
|
Development of Longitudinal Home Activity Datasets as a Shared Resource
|
This project developed portable sensor tools designed to be used in typical homes to collect data for computer science and health research (the BoxLab), as well as to generate shared datasets on home activity from actual homes to be used as a community resource to accelerate research.
Collaborator(s): MIT |
The Boxlab allowed data collection from any home, using portable sensors. The data were aggregated and then could be used for research using this visualization and annotation software.
|
The PlaceLab: A Living Laboratory for Home-Based Technology Research
|
How can ubiquitous computing technologies be created that work for extended periods of time? How can persuasive interfaces designed to motivate behavior change in the home be studied in context? Dr. Intille's research group, in collaboration with partner TIAX, developed the PlaceLab - an apartment-scale shared research facility in a residential condominium building in Cambridge where new technologies and design concepts could be tested and evaluated in the context of everyday living. For a sense of the capabilities it had, watch the PlaceLab overview video, or the more light-hearted video that demonstrates all the sensing. The PlaceLab had to be closed prematurely and was replaced by the BoxLab.
To learn more:
Collaborator(s): MIT, TIAX LLC |
The PlaceLab was a real apartment that had hundreds of sensors embedded within the architecture, including cameras, object usage sensors, current flow sensors, water flow sensors, and more.
Data, including video, audio, and other sensor activation data, were collected and used for experiments, such as automatic recognition of activity.
|
MITes (Portable Object-Usage Sensors)
We developed a kit of wireless portable sensors that could be easily deployed in homes and offices for studying behavior and creating and evaluating novel context-sensitive human-computer interface systems. The sensors were extensively tested (some in the PlaceLab), and they were adopted by a few other research groups for a time. They included sensors for measuring object motion, acceleration, current flow, heart rate, ultraviolet light, indoor position, and more. The mobile version of the sensors evolved into the Wockets.
Computational Perception and Persuasive User Interface Design (multiple projects)
|
Can computers with computational sensing motivate behavior change? Over the years, the group has worked on several projects using mobile and ubiquitous computers to both measure and motivate behavior change. We are studying the intersection between new user interface design strategies, computational tools for studying behavior change, computational learning algorithms for context recognition and planning, and behavioral and social psychology. We focus on the evaluation of algorithms and devices we create in real environments. Much of this work involves measuring and motivating physical activity and healthy eating.
One project developed a "GPS Thermostat" system that may encourage less energy use. Another investigated how point-of-decision prompting in a train station could impact behavior. A third looked at how to motivate less television watching (using a television remote control). |
We use paper prototyping and other user interface design strategies extensively when building prototype systems to deploy in research studies on motivating behavior change.
|
Maternal Stress and Children’s Obesity Risk (MATCH) Study
The goal of this project is to determine whether levels of stress among working mothers are related to increased obesity risk in their children. We are using novel methods such as ecological momentary assessment to examine within-day mother-to-child stress processes that contribute to children’s long-term obesity risk in an accumulated manner over time. Working mothers and their children (9 to 11 years old) are participating in six semi-annual assessments waves across three years using technology created at Northeastern.
Collaborator(s): University of Southern California Medical School
Supported by: NIH/NHLBI
Understanding HIV Risk Environment for Youth in Supportive Housing
Collaborator(s): University of Southern California Medical School
