Assessment
Using the patient centered observation form: Evaluation of an online training program

https://doi.org/10.1016/j.pec.2015.03.005Get rights and content

Highlights

  • An online module trained students and faculty to use a communication assessment tool.

  • Contrasting poor communication with strong communication enhances learning.

  • Teaching students and faculty communication assessment may accelerate learning.

Abstract

Objective

The Patient Centered Observation Form (PCOF) helps trainees identify and describe specific communication skills and enhance self-awareness about skill use. We studied the effectiveness and ease of use of the Improving Communication Assessment Program (ICAP), an online module that prepares trainees to use the PCOF.

Methods

Students, residents and medical educators viewed two videos (common and better skill use) of the same interaction and rated each video using the PCOF. Video sequence was randomized. We assessed agreement with experts, ease of use, concepts learned, and areas of confusion.

Results

Trainees (211) achieved strong agreement (.83) with experts and were highly satisfied (mean 4.18 out of 5). Viewing the common video first produced higher agreement (.87 vs .79; ES = .4) with experts and greater satisfaction (4.36 vs 4.02, ES .4) than viewing the better video first. Trainees reported diverse areas of learning and minimal confusion.

Conclusion

ICAP training to use the PCOF may facilitate teaching and assessment of communication skills and enrich training through peer observation and feedback. We offer several educational strategies.

Practice implications

Learning to use the PCOF via the ICAP module may accelerate communication training for medical students, residents, medical educators and practicing clinicians.

Introduction

Effective communication is an essential physician skill to build trust, convey caring, and provide education and treatment [1], [2], [3], [4]. Most medical schools teach and evaluate communication skills in the first and second years through didactic efforts, simulation, and limited practice with selected patients. However, medical schools are challenged to integrate communication skill training during the clinical phase of training [5]. Faculty often lack the time, training or language to train senior students or residents [6], [7], [8]. Lacking an articulate vocabulary means that educators struggle to reveal their tacit knowledge [8], [9], [10] and instead may rely solely on role modeling without explaining their behavior. A limited vocabulary leads to teachers and learners struggling to teach and articulate questions [7], [8].

The Patient Centered Observation Form (PCOF) was designed to help observers and those being observed identify and describe specific skills, enhance self-awareness [11] and structure coaching during formative assessment. As such our definition of potential trainees includes medical students, residents, nurses, faculty physicians and other medical educators observing others during real patient encounters or on video.

The process of creating the tool began in 2004. The first version of this tool used only narrative feedback about categories in the Kalamazoo consensus statement of essential communication skills [2]. In 2005 behavioral anchors were added corresponding to skills and categories described in the Kalamazoo consensus statement and in 13 communication assessment instruments [12]. The experts contributing to these sources consistently named the selected categories and skills. The tool has been revised several times since 2005 in response to feedback from users. The PCOF (Fig. 1 PCOF 2011 version) categorizes competence, and identifies skills, at the intersection of time management and quality of care [13]. Addressing trainee time management concerns is essential to reduce trainee anxiety that impedes learning [5], [14]. To avoid promoting a grading mentality the PCOF does not use numerical scoring but instead focuses on recording evidence of skill use using a checklist with space to add narrative commentary. Participants categorize the communication skills by checking off one or more elements within each of thirteen PCOF categories. The PCOF requires users to count elements in a skill category to determine competence. Element sums within the range listed in the left-hand column denotes sub-competent performance while element counts in the ranges in either of the two right-hand columns denote competent performance, hence the thick vertical line separating the left hand column from the middle and right-hand column. Not all encounters require element sums in the right hand column.

Research [15], [16] suggests that the training needed to master communication assessment tools requires several hours and is the province of small numbers of highly trained faculty or researchers. However, the PCOF is designed for all trainees including a wide range of medical educators. Therefore we needed a training methodology that is efficient, effective and attractive. In 2007 the Improving Communication Assessment Program (ICAP) was created to provide online asynchronous PCOF training to help users at all developmental levels identify and name core communication skills (http://uwfamilymedicine.org/pcof). The Improving Communication Assessment Program has trainees use the PCOF to rate physician-patient interaction in two videos. One video demonstrates COMMON skill use characterized by the absence of selected skills listed in Table 1. The second, BETTER video, recreates the same interaction but with the physician demonstrating selected skills that were absent in the COMMON version. The trainee downloads a paper version of the PCOF, records observed skills, transfers ratings to ICAP and then submits these ratings. The next ICAP screen shows how experts rated the video1 with an explanation for each rating allowing the user to compare and learn. The training takes approximately 40 min. Using COMMON and BETTER videos helps make communication skills tangible by creating contrast and has been shown to be effective in prior educational trials [5], [17], [18], [19].

The ICAP program is an integral component of the Patient Centered Communication Curriculum embedded within the required six-week, third year family medicine clerkship at the University of Washington School of Medicine (UWSOM). Faculty at one of 51 clerkship sites across our five state region use the ICAP to learn the PCOF as do all the clerkship students (approx. 230/year). Each week clerkship faculty and students are expected to observe one another using the PCOF in an outpatient setting and discuss communication skill use [5].

The ICAP program is also used to train other health care providers in various roles and at various levels of professional development including residents, medical educators, community physicians and nurses. Additionally, It is used for an advanced communication skills course that was disseminated to several medical schools [14].

While we have used consecutive versions of the ICAP for several years, we have not assessed its effectiveness or appeal to users. Therefore, we studied the following questions:

  • 1.

    Is the Improving Communication Assessment Program effective in training users to use the PCOF?

    • a.

      Is there adequate agreement between trainee and expert ratings?

    • b.

      Is the training easy to understand and complete?

    • c.

      Is there a correlation between ease of use and agreement with experts?

  • 2.

    Does the order of viewing the ICAP videos make a significant difference in agreement with the experts and ease of use?

  • 3.

    What are the most important concepts that trainees learn?

  • 4.

    What concepts confuse trainees?

Section snippets

Methods

ICAP programing randomized users to view either the COMMON or BETTER videos first. All responses were anonymous. Prior to viewing videos participants were asked to enter (through typing) age, gender and professional role. After rating both videos participants were asked to describe the “most important concept/skill learned” and concepts “that were confusing”, and rate the ease of using the ICAP training from 1 (extremely difficult to understand and complete) to 5 (very easy to understand and

Results

A total of 211 trainees completed the ICAP training between October 2011 and June 2012. Sixty-five percent were female. Most users (64.5%) were below 30 years of age. The majority were medical students (72%), 8% were Nurses, 4.7% Behavioral Scientists, 4.3% faculty physicians, 1.9% Medical Residents and 7.6% Other. No statistically significant differences were found for comparisons of agreement scores by gender, age (below 30 years of age vs older), and professional role (medical student vs all

Discussion

The Improving Communication Assessment Project (ICAP) (http://uwfamilymedicine.org/pcof) was effective in helping trainees learn to use the Patient Centered Observation Form (PCOF). We found strong agreement between trainee ratings and expert ratings. Users found this online training easy to understand and complete. However, we found no relationship between ease of use and agreement with experts. Training effectiveness appears stronger when rating a less sophisticated skill demonstration before

Conclusion

The Improving Communication Assessment Project helped trainees effectively and efficiently identify and name specific communication skills on the Patient Centered Observation Form. To our knowledge no other communication assessment tool has been evaluated for trainee use or widespread adoption by medical educators. Regularly using the PCOF to observe colleagues during the later years of medical school through residency and into practice may help physicians address long standing struggles to

Role of funding

None.

Conflict of interest statement

Mr. Mauksch receives fees and honoraria for providing training and consultation on communication skills to educational institutions and health care organizations.

Misbah Keen, Jan Carline and Jeanne Cawse-Lucas—None.

Acknowledgements

The authors wish to thank Valerie Ross, MS, Tom Greer, MD, MPH and Frederick Chen MD, MPH for their help creating expert ratings of the ICAP videos.

References (35)

  • T.R. Egnew et al.

    Role modeling the doctor–patient relationship in the clinical curriculum

    Fam Med

    (2011)
  • P.F. Weissmann et al.

    Role modeling humanistic behavior: learning bedside manner from the experts

    Acad Med

    (2006)
  • M.D. Wenrich et al.

    Teachers as learners: the effect of bedside teaching on the clinical skills of clinician-teachers

    Acad Med

    (2011)
  • R.M. Epstein et al.

    Self-monitoring in clinical practice: a challenge for medical educators

    J Contin Educ Health Prof

    (2008)
  • J.M. Schirmer et al.

    Assessing communication competence: a review of current tools

    Fam Med

    (2005)
  • L.B. Mauksch et al.

    Relationship, communication, and efficiency in the medical encounter: creating a clinical model from a literature review

    Arch Intern Med

    (2008)
  • L. Mauksch et al.

    Design, dissemination, and evaluation of an advanced communication elective at seven U.S. medical schools

    Acad Med

    (2013)
  • Cited by (0)

    View full text