Review article
Understanding online health information: Evaluation, tools, and strategies

https://doi.org/10.1016/j.pec.2016.08.028Get rights and content

Highlights

  • Assessing online health material is a central issue in patient education.

  • Readability tools used in evaluation of online health material are covariant.

  • The reliability of tools differs between readability and understandability.

  • Combined quantitative/qualitative approaches optimize assessment strategies.

Abstract

Objective

Considering the status of the Internet as a prominent source of health information, assessing online health material has become a central issue in patient education. We describe the strategies available to evaluate the characteristics of online health information, including readability, emotional content, understandability, usability.

Methods

Popular tools used in assessment of readability, emotional content and comprehensibility of online health information were reviewed. Tools designed to evaluate both printed and online material were considered.

Results

Readability tools are widely used in online health material evaluation and are highly covariant. Assessment of emotional content of online health-related communications via sentiment analysis tools is becoming more popular. Understandability and usability tools have been developed specifically for health-related material, but each tool has important limitations and has been tested on a limited number of health issues.

Conclusion

Despite the availability of numerous assessment tools, their overall reliability differs between readability (high) and understandability (low). Approaches combining multiple assessment tools and involving both quantitative and qualitative observations would optimize assessment strategies.

Practice implications

Effective assessment of online health information should rely on mixed strategies combining quantitative and qualitative evaluations. Assessment tools should be selected according to their functional properties and compatibility with target material.

Introduction

As Internet-based digital media have become more integrated into everyday life, popular information consumption behavior and strategies have shifted from a traditional model to a digital model [1]. In contrast to the traditional model in which information was obtained through direct contact with reliable information sources, such as teachers, professionals, or printed material, information retrieval in the online era is characterized by indirect connections, multiplicity of sources, and low levels of reliability [1], [2], [3], [4].

While this shift has occurred in every domain of human knowledge, it has particular implications for the dissemination of health-related information. Online sources of health-related information require particular oversight to ensure that they provide accurate, appropriate and understandable content that meets the unique needs of various populations of patients. In other words, the publication of online materials is not useful if they are not beneficial for patients. Furthermore, in the context of online resources such as forums and blogs − and despite the involvement of the community of users and moderators − information can be uncontrolled and sometimes poorly moderated, leading to the risk of spreading potentially harmful information [5].

Health literacy can be defined as the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions [6]. This concept of health literacy not only implies the ability of patients to seek and understand health information, but also involves the capacity to evaluate and to use the information in order to make sound decisions about health-related issues [6], [7]. While essential in the context of traditional media, health literacy is obviously also central to the construction of Internet-based health resources and interventions [8]. Of particular concern is evidence suggesting that health literacy is relatively low in the general population [9], [10], [11], which is associated with poorer use of health care services and poorer health outcomes [12], [13]. This also contributes in part to health disparities [14]. Not surprisingly, the majority of patient health education materials, including those found online, surpass the recommended readability level suggested by the US National Institutes of Health [5], [15], [16], [17], [18], [19]. When confronted with non-adapted materials, patients might shift to sources which they would consider understandable, without these sources being necessarily endorsed by public health authorities. For instance, numerous patients consider websites such as Wikipedia as a primary source of information regarding their medical status or the different treatment options [20].

Although the health literacy of a given population can be measured, evaluating the appropriateness and understandability of online health-related material poses a challenge. Online material is composed of multiple elements that need to be considered. Some of these elements are shared with offline material, such as the validity and reliability of the information itself, the characteristics of the text (e.g., readability), the semantic complexity, and the way the content is organized and presented. In contrast, other elements are specific to online communication modes, such as the emotional tone (which is considerably more salient in user-generated content) and the use of multimedia materials. All of these elements contribute to the overall understandability and potential usability of the content. The need for evaluation of the appropriateness of online content has been recognized early on and a number of tools to evaluate online informational materials have been developed over the past two decades. However, given the complexity of the online material, there is currently no single tool that can be used to evaluate all of the various elements of health-related online content. Therefore, evaluation of online health material requires a combination of various methodologies, ranging from classical readability tools to more experimental medical content assessment questionnaires. This paper describes the available tools for assessment of online content and proposes that combined (quantitative and qualitative) approaches could be successfully used to evaluate online health information based on several complementary dimensions including readability, emotional tone, understandability and usability, connectivity and multimodality of content.

Section snippets

Readability: the ease of reading of online health information

When attempting to assess any form of text-based information, the first and obvious parameter to evaluate is whether the text is linguistically understandable − in other words, its readability. Quantifying readability is not a novel issue: over the last half-century, numerous tools have been developed and are available to assess the level of readability of text [21]. These tools have been used abundantly in order to evaluate professionally generated online medical materials [5], [15], [16], [17]

Emotional tone: potential emotional impact of online health information

In contrast to conventional media in which the degree of interaction between users and the content is limited, online applications where users have a more active role, such as social media, permit the introduction of additional informational elements into the content. In particular, interactions between users in online discussions of health-related information can introduce emotional elements that may impact the subsequent interpretation of the content by the users [28]. Furthermore, as

Understandability and usability: empowering the reader

While linguistic and semantic issues are obviously crucial when it comes to comprehending text-based content, they are however insufficient to fully define the understandability of given material. This is particularly true in the context of health-related information, in which characteristics such as suitability and reliability of the information are of central importance in properly conveying the desired message and ultimately leading to usability of the presented information. Therefore, in

Combined approaches and innovative research directions

Despite the multiplicity of available instruments, several critical problems remain unaddressed. First, most if not all evaluation tools − whether readability, sentiment analysis, or understandability and usability assessment tools − have been used on online materials covering a limited number of health issues, which prevents large-scale generalization of the utility of these tools. Second, critical studies directly comparing the results of analyses using different tools are needed. The few

Conclusion

Online resources offer a wide range of possibilities for health professionals to enhance patients’ health literacy. The numerous tools that we have discussed demonstrate that assessing online health information is possible. However, to be done effectively, several elements need to be considered. First, a thorough evaluation of online content requires the exploration of several dimensions (among which are readability, emotional tone, understandability and usability), Second, the characteristics

Acknowledgements

MJG holds a Career Grant from the “Fonds de Recherche du Québec − Santé” (FRQS). This work was supported by the Oto-Rhino-Laryngology − Head and Neck Surgery Research Fund of the “Fondation de l'Université Laval”.

References (77)

  • A.M. Lomanowska et al.

    Online intimacy and well-being in the digital age

    Internet Interv.

    (2016)
  • T.O. Blank et al.

    Differences among breast and prostate cancer online support groups

    Comput. Hum. Behav.

    (2010)
  • R.E. Guadagno et al.

    Even in virtual environments vomen shop and men build: a social role perspective on second life

    Comput. Hum. Behav.

    (2011)
  • K. Denecke et al.

    Sentiment analysis in medical settings: new opportunities and challenges

    Artif. Intell. Med.

    (2015)
  • G.W. Alpers et al.

    Evaluation of computerized text analysis in an Internet breast cancer support group

    Comput. Hum. Behav.

    (2005)
  • U. Pfeil et al.

    Age differences in online social networking: a study of user profiles and the social capital devide among teenagers and older users in MySpace

    Comput. Hum. Behav.

    (2009)
  • K.R. Hexem et al.

    Putting on a happy face: emotional expression in parents of children with serious illness

    J. Pain Symptom Manage.

    (2013)
  • R.G. Rodrigues et al.

    SentiHealth-Cancer: a sentiment analysis tool to help detecting moodof patients in online social networks

    Int. J. Med. Inform.

    (2016)
  • N. Corcoran et al.

    The readability and suitability of sexual health promotion leaflets

    Patient Educ. Couns.

    (2016)
  • S.J. Shoemaker et al.

    Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient information

    Patient Educ. Couns.

    (2014)
  • M.J. Bernier

    Establishing the psychometric properties of a scale for evaluating quality in printed education materials

    Patient Educ. Couns.

    (1996)
  • C. Cerminara et al.

    Use of the DISCERN tool for evaluating web searches in childhood epilepsy

    Epilepsy Behav.

    (2014)
  • F. Smith

    Readability, suitability and comprehensibility in patient education materials for Swedish patients with colorectal cancer undergoing elective surgery: a mixed method design

    Patient Educ. Couns.

    (2014)
  • S.G. Smith et al.

    ABCs or 123s? The independent contributions of literacy and numeracy skills on health task performance among older adults

    Patient Educ. Couns.

    (2015)
  • C. Zellmer et al.

    Evaluating the usefulness of patient education materials on surgical site infection: a systematic assessment

    Am. J. Infect. Control

    (2015)
  • A.M. Lomanowska et al.

    My avatar is pregnant! Representation of pregnancy, birth, and maternity in a virtual world

    Comput. Hum. Behav.

    (2014)
  • N. Diviani et al.

    Exploring the role of health literacy in the evaluation of online health information: insights from a mixed-methods study

    Patient Educ Couns.

    (2016)
  • M.J. Guitton

    The immersive impact of meta-media in a virtual world

    Comput. Hum. Behav.

    (2012)
  • D. Brossard

    New media landscapes and the science information consumer

    Proc. Natl. Acad. Sci. U. S. A.

    (2013)
  • K. Sorensen et al.

    Health Literacy and public health: a systematic review and integration of definitions and models

    BMC Public Health

    (2012)
  • E. Sillence et al.

    How do patients evaluate and make use of online information?

    Soc. Sci. Med.

    (2007)
  • M. Mackert et al.

    EHealth and health literacy: a research methodology review

    J. Comput. Mediat. Commun.

    (2014)
  • H. Levy et al.

    Health literacy and the digital device among older americans

    J. Gen. Intern. Med.

    (2015)
  • V. Duren-Winfield et al.

    Health literacy and computer-assisted instruction: usability and patient preference

    J. Health Commun.

    (2015)
  • H.S. Yin et al.

    The health literacy of parents in the United States: a nationally representative study

    Pediatrics

    (2009)
  • N. Berkman et al.

    Low health literacy and health outcomes: an updated systematic review

    Ann. Intern. Med.

    (2011)
  • D.A. DeWalt et al.

    Literacy and health outcomes: A systematic review of the literature

    J. Gen. Intern. Med.

    (2004)
  • T.L. Sentell et al.

    Importance of adult literacy in understanding health disparities

    J. Gen. Intern. Med.

    (2006)
  • Cited by (0)

    View full text