Establishing the psychometric properties of a scale for evaluating quality in printed education materials

doi:10.1016/S0738-3991(96)00927-5

Patient Education and Counseling

Volume 29, Issue 3, December 1996, Pages 283-299

https://doi.org/10.1016/S0738-3991(96)00927-5 Get rights and content

Abstract

A study establishing the psychometric properties of a quality standard for quantifying the presence (or absence) of instructional design/learning principles contained in printed education materials (PEMs) was conducted in 4 phases. Content validity, a pretest which included creation of a rating key, inter-rater reliability, and intra-rater reliability levels were established for the Bernier Instructional Design Scale (BIDS). Experts in patient education established content validity by identifying 37 instructional design/learning principles as essential to PEM quality from a 90 item domain. An inter-disciplinary group of 89 health professionals applied the BIDS to a test PEM. Group ratings were compared with a rating key which served as a validated standard for the presence of principles in the test PEM. Reliability of the BIDS achieved acceptable levels when rating categories were collapsed from 4 to 3 levels to be consistent with a measurement model for rating checklists. A revised methodological approach and program of psychometric research are described.

References (64)

BE Giloth
Management of patient education in US hospitals: evolution of a concept
Patient Educ Couns
(1990)
MJ Bernier et al.
Designing and evaluating printed education materials: model and instrument development
Patient Educ Couns
(1991)
M Frank-Stromborg et al.
Evaluating written patient education materials
Semin Oncol Nurs
(1991)
MT Oberst
Perspectives on research in patient teaching
Nurs Clin North Am
(1989)
J Wilson-Barnett et al.
Studies evaluating patient teaching: implications for practice
Int J Nurs Studies
(1983)
GC Baker et al.
Increased readability improves comprehension of written information for patients with skin diseases
J Acad Dermatol
(1988)
JC Reid et al.
Why people don't learn from diabetes literature: influence of text and reader characteristics
Patient Educ Couns
(1995)
E Roux
Developing an educational booklet about back pain: problems encountered — possible solutions
Patient Educ Couns
(1995)
LG Frederikson et al.
Evaluation of a patient education leaflet designed to improve communication in medical consultations
Patient Educ Couns
(1995)
K Parrinello
Patients' evaluation of a teaching booklet for arterial bypass surgery
Patient Educ Couns
(1984)

AB Zion

The process of developing patient education materials for infertile couples

J Obstet Gynecol Neonatal Nurs

(1988)

LW Green et al.

Measurement and evaluation in health promotion

(1986)

SA Brown

Quality of reporting in diabetes patient education research

Res Nur Health

(1990)

S Kruger

A review of patient education in nursing

J Nurs Staff Dev

(1990)

MJ Lipetz et al.

What is wrong with patient education programs?

Nurs Outlook

(1990)

D Allensworth et al.

Evaluating printed materials

Nurse Educ

(1986)

MJ Bernier

Developing and evaluating printed education materials: a prescriptive model for quality

Orthop Nurs

(1993)

MD Boyd

A guide to writing effective patient education materials

Nurs Manage

(1989)

RG Brockett

Developing written learning materials: a proactive approach

Lifelong Learn

(1984)

BG Cox

The art of writing patient education materials

Prof Med Assist

(1989)

L Dalton et al.

Evaluating printed health information for consumers

Bull Med Libr Assoc

(1981)

P Farrell-Miller et al.

How effective are your patient education materials? Guidelines for developing and evaluating written education materials

Diabetes Educ

(1989)

SD Foster

Written reinforcement for teaching

Matern Child Nurs J

(1986)

JW Lange

Developing printed materials for patient education

Dimens Crit Care Nurs

(1989)

G Lohr et al.

An experience in designing patient education materials

J Nurs Staff Dev

(1989)

RA Lorenz et al.

Evaluation of education program developments: illustration of the research development cycle

Diabetes Educ

(1989)

DT Manning

Communicating messages via print media

Occup Health Saf

(1981)

DB Mathis

Writing patient education materials

Orthop Nurs

(1989)

BJ McCabe

A strategy for designing effective patient education materials

J Am Diet Assoc

(1989)

SH Rankin et al.

Patient education: issues, principles and guidelines

(1983)

M Rice et al.

A simple guide for the design, use, and evaluation of education materials

Health Educ Q

(1991)

L Rohert et al.

Effective use of patient education illustrations

Patient Educ Couns

(1991)

Cited by (28)

In what ways does user experience design improve printed educational materials?
2023, Applied Ergonomics
We report on a series of four studies that investigated how user experience design (UXD) can improve printed educational materials (PEMs). We examined the perceived usability of an existing PEM for breast cancer screening and observed the usability problems associated with it (Study 1). We then compared a breast cancer screening PEM created by user experience designers with two other breast cancer screening PEMS, finding that the PEM based on UXD had higher perceived usability, and lower mentions of usability problems, than the other two PEMs (Study 2). We next examined the impact of individual differences in design expertise on perceived usability, this time including a PEM on cervical cancer screening as well as one on breast cancer screening (Study 3). Our concluding study (Study 4) then examined the impacts of UXD on learnability of PEM content as defined by answers to a knowledge questionnaire about screening administered before and after reading the PEM, and by intention to screen for cancer after reading the PEM. The first three studies showed that the involvement of UXD improved the perceived usability of PEMs, and Study 3 showed that designers differ in their ability to create useable PEMs. Study 4 failed to find a corresponding improvement in learnability or intention to screen when UXD was used to improve perceived usability. We conclude that a user experience design approach that incorporates graphic design can improve the perceived usability of PEMs in some situations (e.g., when the PEM material is not too lengthy or complex, and when the graphic designer is sufficiently skilled). However, we found no evidence that lack of perceived usability accounted for the failure of PEMS (found in previous research) to improve knowledge or intention to screen.
Understanding user acceptance of battery swapping service of sustainable transport: An empirical study of a battery swap station for electric scooters, Taiwan
2020, International Journal of Sustainable Transportation
A battery swapping service is used to address the challenges associated with replacing scooters with electric two wheelers (E2Ws) in the Taiwanese two-wheeler market. In this study, two experiments were conducted to compare two different battery swap stations (BSSs) and to further explore usability, user experiences (UXs), and motivation in order to determine how the service affects the quality evaluation made by the users on the basis of their interactions. The experiments were administered to 140 participants of experiencing BSS services located in the city of New Taipei. The results verified that both BSS can help users to successfully accomplish their goal of swapping batteries. Furthermore, the design and usage of the G-BSS makes a positive impression on users and yields an acceptance rate that exceeds 80% and stimulates ∼32.9% of potential users to accommodate E2Ws. This study concluded that good esthetic quality should include design beauty and the design that could create a good UX for the users. Aesthetic quality and hedonic quality were positively correlated with intrinsic motivation and user satisfaction. These may further influence user’s continuance usage intention. Furthermore, an important factor that influences pragmatic quality is whether the user’s perception of the product is established during the course of using the product. Based on these results, a number of modifications to BSS are suggested for improving UXs in the context of battery swapping system. It is recommended that designers and developers should consider the above issues when implementing newly versions of battery swapping services.
Understanding online health information: Evaluation, tools, and strategies
2017, Patient Education and Counseling
Citation Excerpt :
Another scale developed the same year as the SAM, the Bernier Instructional Design Scale (BIDS), aims at quantifying the instructional design/learning principles contained in printed education materials [61]. However, the inter-judge reliability of the scale is poor [57,61] and it does not fully take into account the visual aspects (particularly layouts and graphics) of the material [57]. Both the SAM and the BIDS share common limitations: while both take into account some graphical aspects of the material, they were primarily designed to assess printed material, which differs significantly from online material.
Considering the status of the Internet as a prominent source of health information, assessing online health material has become a central issue in patient education. We describe the strategies available to evaluate the characteristics of online health information, including readability, emotional content, understandability, usability.
Popular tools used in assessment of readability, emotional content and comprehensibility of online health information were reviewed. Tools designed to evaluate both printed and online material were considered.
Readability tools are widely used in online health material evaluation and are highly covariant. Assessment of emotional content of online health-related communications via sentiment analysis tools is becoming more popular. Understandability and usability tools have been developed specifically for health-related material, but each tool has important limitations and has been tested on a limited number of health issues.
Despite the availability of numerous assessment tools, their overall reliability differs between readability (high) and understandability (low). Approaches combining multiple assessment tools and involving both quantitative and qualitative observations would optimize assessment strategies.
Effective assessment of online health information should rely on mixed strategies combining quantitative and qualitative evaluations. Assessment tools should be selected according to their functional properties and compatibility with target material.
Evaluation of written patient educational materials in the field of diagnostic imaging
2009, Radiography
Citation Excerpt :
In terms of instructiveness, the purpose of the materials was missing. Instructive examples make it easier to find the interesting things related to a patient's prevailing situation.9–11 For instance, the patient is provided with information about their diet before the examination or how to dress for the magnetic examination.
To evaluate the quality of written educational materials for diagnostic imaging (radiological and nuclear medicine) patients.
Written educational materials (n = 70) for diagnostic imaging patients were analysed. The materials were evaluated based on their external appearance (9 criteria), instructiveness (7), content (7), language and structure (8) and readability (1). Deductive content analysis was used. Quantified parts of the analyses were analysed by SAS for Windows. Dependence between criteria (32) was tested by Pearson correlation coefficients.
The external appearance fulfilled almost completely the criteria of good written education materials. The instructiveness was addressed clearly, except for the purpose of the material. The contents of materials dealt with bio-physiological, functional and cognitive dimensions of knowledge, while financial dimensions of knowledge were hardly dealt with at all. The language and the structure were reasonably good, but the language was partly in passive voice and the text contained strange words. Most of the education material was moderately easy to read.
The results show that the quality of material was quite good in all dimensions. Only a small number of criteria were unsatisfactory. The results can be used to further improve written patient education materials and patient education in the imaging unit.
Promoting health literacy
2005, Journal of the American Medical Informatics Association
Citation Excerpt :
Some investigators have been concerned about the cultural appropriateness of patient materials, particularly with regard to health beliefs.67–75 Recently, some have begun to look at the readability of health information posted on the Internet, but this research is in its early stages.76–80 A variety of techniques and checklists have been suggested for improving the accessibility of materials created for patients. (
This report reviews some of the extensive literature in health literacy, much of it focused on the intersection of low literacy and the understanding of basic health care information. Several articles describe methods for assessing health literacy as well as methods for assessing the readability of texts, although generally these latter have not been developed with health materials in mind. Other studies have looked more closely at the mismatch between patients' literacy levels and the readability of materials intended for use by those patients. A number of studies have investigated the phenomenon of literacy from the perspective of patients' interactions in the health care setting, the disenfranchisement of some patients because of their low literacy skills, the difficulty some patients have in navigating the health care system, the quality of the communication between doctors and their patients including the cultural overlay of such exchanges, and ultimately the effect of low literacy on health outcomes. Finally, the impact of new information technologies has been studied by a number of investigators. There remain many opportunities for conducting further research to gain a better understanding of the complex interactions between general literacy, health literacy, information technologies, and the existing health care infrastructure.
Written orthopedic patient education materials from the point of view of empowerment by education
2004, Patient Education and Counseling
Written patient education materials are one way of supporting patient empowerment. The aim of this study was to evaluate written orthopedic patient education materials (n=25) drawn from a university hospital’s electronic databank. In the absence of suitable tools for evaluation, an analytic framework was developed by a panel of nursing scientists and clinical experts. The materials were evaluated from the point of view of empowerment by their external appearance, content and instructiveness. In terms of their external appearance the materials were well prepared, but much was left to be desired with regard to contents and instructiveness. Development needs to focus on making better use of methods of visual representation, on increasing the coverage of content areas that so far have received less attention, like social, experiential, ethical and financial empowerment, and on providing a more focused perspective on the patient.

View all citing articles on Scopus

^∗: Current address, 38 Colony Park Circle, Galveston, TX 77551, USA. Tel: +1 409 744 0166; Fax: + 1 409 772 9598 (Dean of Medicine Office).

View full text

Establishing the psychometric properties of a scale for evaluating quality in printed education materials

Abstract

Patient Educ Couns

Patient Educ Couns

Semin Oncol Nurs

Nurs Clin North Am

Int J Nurs Studies

J Acad Dermatol

Patient Educ Couns

Patient Educ Couns

Patient Educ Couns

Patient Educ Couns

J Obstet Gynecol Neonatal Nurs

Measurement and evaluation in health promotion

Quality of reporting in diabetes patient education research

Res Nur Health

A review of patient education in nursing

J Nurs Staff Dev

What is wrong with patient education programs?

Nurs Outlook

Evaluating printed materials

Nurse Educ

Developing and evaluating printed education materials: a prescriptive model for quality

Orthop Nurs

A guide to writing effective patient education materials

Nurs Manage

Developing written learning materials: a proactive approach

Lifelong Learn

The art of writing patient education materials

Prof Med Assist

Evaluating printed health information for consumers

Bull Med Libr Assoc

How effective are your patient education materials? Guidelines for developing and evaluating written education materials

Diabetes Educ

Written reinforcement for teaching

Matern Child Nurs J

Developing printed materials for patient education

Dimens Crit Care Nurs

An experience in designing patient education materials

J Nurs Staff Dev

Evaluation of education program developments: illustration of the research development cycle

Diabetes Educ

Communicating messages via print media

Occup Health Saf

Writing patient education materials

Orthop Nurs

A strategy for designing effective patient education materials

J Am Diet Assoc

Patient education: issues, principles and guidelines

A simple guide for the design, use, and evaluation of education materials

Health Educ Q

Effective use of patient education illustrations

Patient Educ Couns