Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient information
Introduction
Health literacy is the capacity to “obtain, process and understand basic health information and services needed to make appropriate health decisions” [1]. Health literacy is well recognized as a challenge for public health, with many adults lacking the requisite skills to engage successfully in their health care. Recent systematic reviews have confirmed that low health literacy is strongly associated with poorer use of health care and subsequent health outcomes, leading to higher use of emergency departments and inpatient beds [2], [3], [4].
While the skills of individuals are an important part of health literacy, the field has come to recognize that the demands placed on individuals by the health system and professionals are an important determinant [5], [6], [7], [8], [9]. To address health literacy, the U.S. Department of Health and Human Services’ National Action Plan to Improve Health Literacy (National Action Plan) promotes a multisector effort to improve health literacy, including reducing the demands placed on individuals [10]. A key goal of the National Action Plan is “to develop and disseminate health and safety information that is accurate, accessible, and actionable” [10]. Studies assessing the readability, suitability or comprehensibility of patient education materials on a myriad of topics are abound, and the evidence is clear that most education materials are too complex for patients with low health literacy.
Organizations and professionals aspiring to produce low-demand patient education materials have a selection of guides that provide instruction [11], [12], [13], [14], [15]. Readability formulas [13], [16] are commonly relied upon to assess whether written materials are in fact low-demand. Readability formulas provide quantitative estimates, in the form of a grade level, of the reading difficulty of written information based on word and sentence difficulty. Yet readability formulas ignore several factors that contribute to comprehension [13].
In recognition of the shortcomings of readability formulas, several checklists and instruments that assess the health literacy demand of materials have been developed [17], [18], [19], [20], [21], [22], [23], including two newly-developed instruments [24], [25]. These assessment tools, however, have not shown inter-rater reliability [17], were developed or tested with a specific topic or aim [18], [19], [20], or were tested using only raters trained in the use of the instrument [24], [25]. Furthermore, most are applicable only to print material, and none measure whether materials are actionable – an important characteristic of materials called for in the first goal of the National Action Plan [10].
The aim of this study was to develop a reliable and valid Patient Education Materials Assessment Tool (PEMAT) to be used by untrained lay and health professionals alike to assess the understandability and actionability of both printable (e.g., printed materials like brochures or pamphlets or materials that can be printed from websites like PDFs) and audiovisual (e.g., video or multi-media presentation with or without narration) patient education materials. We defined understandability and actionability as follows:
Understandability: Patient education materials are understandable when consumers of diverse backgrounds and varying levels of health literacy can process and explain key messages.
Actionability: Patient education materials are actionable when consumers of diverse backgrounds and varying levels of health literacy can identify what they can do based on the information presented.
Section snippets
Methods
The PEMAT was iteratively and systematically developed with repeated input from a panel of experts in health literacy; health communications; content creation, including different modalities; patient education; communication; patient engagement, and health information technology. The experts included clinicians, researchers, policymakers, academicians, and staff from non-profit, for-profit and governmental organizations. (See Acknowledgements for the list of experts.)
Once we defined
Review of existing instruments and guides
From an initially identified set of 41 instruments and guides, 22 had potentially relevant concepts and items for understandability or actionability. From these instruments, 7 topics (i.e., content, word choice and style, numeracy, quality visual aids, organization, layout and design, ease of use for audiovisual materials only) and 64 associated items were initially compiled. The project team reviewed and narrowed the pool of items based on relevance to understandability or actionability and
Discussion
This article reports on the development and reliability and validity testing of the Patient Education Materials Assessment Tool (PEMAT), a new instrument to assess the understandability and actionability of both printable and audiovisual patient education materials on diverse topics. The development of the PEMAT was guided by earlier work defining and measuring desirable characteristics of patient education materials and a multidisciplinary expert panel. The PEMAT had strong internal
Funding
The information upon which this publication is based was performed under Contract #HHSA290200900012I, TO 4 “Improving EHRs Patient Education Materials” funded by the Agency for Healthcare Research and Quality (AHRQ), Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does the mention of trade names, commercial products, or organizations imply endorsement by the U.S.
Conflicts of interest
All authors state that there are no other actual or potential conflicts of interest including any financial, personal or other relationships with other people or organizations within 3 years of beginning the submitted work that could inappropriately influence, or be perceived to influence, their work.
Acknowledgments
We would like to acknowledge the raters from Abt Associates, AHRQ, Massachusetts General Hospital, and Northwestern University who helped to establish the reliability of the PEMAT, Allyson Ross Davies for her guidance on instrument development, and Ken Carlson and Mark Spranca from Abt Associates for their valuable engagement with the reliability and validity testing of the PEMAT.
We would like to thank the technical expert panel who helped to shape this instrument by providing guidance and
References (37)
- et al.
High agreement but low kappa: I. The problems of two paradoxes
J Clin Epidemiol
(1990) Healthy people 2010
(2000)- et al.
Low health literacy and health outcomes: an updated systematic review
Ann Intern Med
(2011) - et al.
Literacy and health outcomes: a systematic review of the literature
J Gen Intern Med
(2004) - et al.
Impact of health literacy on health outcomes in ambulatory care patients: a systematic review
Ann Pharmacother
(2008) Research-based web design and usability guidelines
(2006)- et al.
Ten attributes of a health literate health care organization
(2012) The meaning and the measure of health literacy
J Gen Intern Med
(2006)Objective 11-2. Improvement of health literacy
Communicating health: priorities and strategies for progress
(2003)
National action plan to improve health literacy
Pfizer principles for clear health communication
Quick guide to health literacy
The toolkit for making written material clear and effective
Clear & simple: developing effective print materials for low-literate readers
How to write easy-to-read health materials
A systematic review of readability and comprehension instruments used for print and web-based cancer information
Health Educ Behav
Teaching patients with low literacy skills
Cited by (517)
Quality assessment of health science-related short videos on TikTok: A scoping review
2024, International Journal of Medical InformaticsA health literacy analysis of online patient-directed educational materials about mycobacterium avium complex
2024, Journal of Clinical Tuberculosis and Other Mycobacterial DiseasesEvaluation of postoperative ophthalmology patient instructions from ChatGPT and Google Search
2024, Canadian Journal of Ophthalmology