The mHealth Data Gap… Results TBD

The Background

The last few decades have seen an explosion in the ubiquity of access to cell phones, not only in first world nations, but on a global scale. According to a 2013 report by Pew Global, many emerging and developing nations have essentially “skipped” using landlines and gone straight to widespread use of cellular phones – likely due to their ever-increasing affordability and availability. Among these developing nations, the median percentage of adults owning a cell phone is 83%!

The emergence of cell phone use has led to a trend in using these technologies to support health objectives – a movement broadly known as mHealth. This broad category can encompass a variety of health efforts ranging from telemedicine to data collection to health promotion and beyond. A 2011 report by the WHO documented the expansion of this movement, reporting that 83% of member states offered at least one mHealth service.

Flickr - Mark Kelley

Flickr – Mark Kelley













The Problem

Despite the fact that mHealth interventions have been widely implemented and studied, the effectiveness of these interventions has not been adequately evaluated or documented. For example, the aforementioned WHO 2011 report indicated that only 12% of the member states evaluated the performance of mHealth services. This is a problem because mHealth initiatives compete for health dollars with other interventions; thus to begin scaling up mHealth funding, there must first be empirical evidence of the relative value and efficiency of these programs. Since 2011, several other studies have examined trends in the evaluation of mHealth programs, and found similar shortcomings.

In 2012, Gurman et al. published a systematic review of the literature on mHealth effectiveness in developing countries. The authors reviewed 44 studies on behavior change communication (BCC) mHealth interventions in developing countries, evaluated their strength, and analyzed their findings. Only 16 of these studies fit the author’s criteria for rigor and quality, and only 5 were peer-reviewed. Furthermore, only 2 of the studies implemented a method long-term evaluation of the effectiveness of the program. The authors recommend that future mHealth interventions must incorporate a plan for long-term and rigorous evaluation of results in accordance with best practices in the published literature.

In 2013, Tomlinson et al. echoed concerns about the mHealth data gap in their article titled Scaling up mHealth: Where is the Evidence? The article notes that despite “hundreds of mHealth pilot studies, there has been insufficient programmatic evidence to inform implementation and scale-up of mHealth” (p. 2). Many of the existing studies are not of high academic quality and conducted by entrepreneurs and researchers with a vested interest in demonstrating the efficacy of their program; meanwhile the few randomized clinical trials of mHealth interventions have shown mixed and modest results. The authors recommend detailed new frameworks for implementing and evaluating future mHealth programs with the aim of providing informative and academically rigorous data on their effectiveness.

In response to the concern about the mHealth data gap, a group of researchers gathered at the mHealth evidence workshop at NIH in 2013 (Kumar et al. 2013). Discussions were had about the reliability and validity of mHealth data. Participants agreed that mHealth studies present unique challenges in obtaining reliable and valid data. For example, mHealth interventions have difficulty establishing convergent validity because they are inherently novel and have no established gold standard for comparisons of effectiveness. The authors ultimately proposed several potential research designs to overcome these obstacles.

The Takeaway

The important lesson for public health professionals in the mHealth arena is to be cognizant of this data gap. There is a scientific responsibility to be aware of the literature and best practices in this area before implementing a new mHealth initiative. It is critical to use academically rigorous methods to validate results and to incorporate funding for long-term follow-up into proposals. Only then will we be able to develop, fund, and scale-up mHealth initiatives that have maximal impact.



Tilly A. Gurman , Sara E. Rubin & Amira A. Roess (2012) Effectiveness of mHealth Behavior Change Communication Interventions in Developing Countries: A Systematic Review of the Literature, Journal of Health Communication: International Perspectives, 17:sup1, 82-104, DOI:10.1080/10810730.2011.649160

Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC (2013) Scaling Up mHealth: Where Is the
Evidence? PLoS Med 10(2): e1001382. doi:10.1371/journal.pmed.1001382

Kumar S, WJ Nilsen, A Abernethy, A Atienza, K Patrick, M Pavel, W Riley, A Shar, B Spring, D Spruijt-Metz, D Hedeker, V Honavar, R Kravitz, RC Lefebvre, DC Mohr, SA Murphy, C Quinn, V Shusterman, D Swendeman, Mobile Health Technology Evaluation: The mHealth Evidence Workshop, American Journal of Preventive Medicine, Volume 45, Issue 2, August 2013, Pages 228-236, ISSN 0749-3797,


One thought on “The mHealth Data Gap… Results TBD

  1. Love the “take away” comment. Success of mhealth depends on a”cademically rigorous methods to validate results and to incorporate funding.”

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s