AssessmentUsing the patient centered observation form: Evaluation of an online training program
Introduction
Effective communication is an essential physician skill to build trust, convey caring, and provide education and treatment [1], [2], [3], [4]. Most medical schools teach and evaluate communication skills in the first and second years through didactic efforts, simulation, and limited practice with selected patients. However, medical schools are challenged to integrate communication skill training during the clinical phase of training [5]. Faculty often lack the time, training or language to train senior students or residents [6], [7], [8]. Lacking an articulate vocabulary means that educators struggle to reveal their tacit knowledge [8], [9], [10] and instead may rely solely on role modeling without explaining their behavior. A limited vocabulary leads to teachers and learners struggling to teach and articulate questions [7], [8].
The Patient Centered Observation Form (PCOF) was designed to help observers and those being observed identify and describe specific skills, enhance self-awareness [11] and structure coaching during formative assessment. As such our definition of potential trainees includes medical students, residents, nurses, faculty physicians and other medical educators observing others during real patient encounters or on video.
The process of creating the tool began in 2004. The first version of this tool used only narrative feedback about categories in the Kalamazoo consensus statement of essential communication skills [2]. In 2005 behavioral anchors were added corresponding to skills and categories described in the Kalamazoo consensus statement and in 13 communication assessment instruments [12]. The experts contributing to these sources consistently named the selected categories and skills. The tool has been revised several times since 2005 in response to feedback from users. The PCOF (Fig. 1 PCOF 2011 version) categorizes competence, and identifies skills, at the intersection of time management and quality of care [13]. Addressing trainee time management concerns is essential to reduce trainee anxiety that impedes learning [5], [14]. To avoid promoting a grading mentality the PCOF does not use numerical scoring but instead focuses on recording evidence of skill use using a checklist with space to add narrative commentary. Participants categorize the communication skills by checking off one or more elements within each of thirteen PCOF categories. The PCOF requires users to count elements in a skill category to determine competence. Element sums within the range listed in the left-hand column denotes sub-competent performance while element counts in the ranges in either of the two right-hand columns denote competent performance, hence the thick vertical line separating the left hand column from the middle and right-hand column. Not all encounters require element sums in the right hand column.
Research [15], [16] suggests that the training needed to master communication assessment tools requires several hours and is the province of small numbers of highly trained faculty or researchers. However, the PCOF is designed for all trainees including a wide range of medical educators. Therefore we needed a training methodology that is efficient, effective and attractive. In 2007 the Improving Communication Assessment Program (ICAP) was created to provide online asynchronous PCOF training to help users at all developmental levels identify and name core communication skills (http://uwfamilymedicine.org/pcof). The Improving Communication Assessment Program has trainees use the PCOF to rate physician-patient interaction in two videos. One video demonstrates COMMON skill use characterized by the absence of selected skills listed in Table 1. The second, BETTER video, recreates the same interaction but with the physician demonstrating selected skills that were absent in the COMMON version. The trainee downloads a paper version of the PCOF, records observed skills, transfers ratings to ICAP and then submits these ratings. The next ICAP screen shows how experts rated the video1 with an explanation for each rating allowing the user to compare and learn. The training takes approximately 40 min. Using COMMON and BETTER videos helps make communication skills tangible by creating contrast and has been shown to be effective in prior educational trials [5], [17], [18], [19].
The ICAP program is an integral component of the Patient Centered Communication Curriculum embedded within the required six-week, third year family medicine clerkship at the University of Washington School of Medicine (UWSOM). Faculty at one of 51 clerkship sites across our five state region use the ICAP to learn the PCOF as do all the clerkship students (approx. 230/year). Each week clerkship faculty and students are expected to observe one another using the PCOF in an outpatient setting and discuss communication skill use [5].
The ICAP program is also used to train other health care providers in various roles and at various levels of professional development including residents, medical educators, community physicians and nurses. Additionally, It is used for an advanced communication skills course that was disseminated to several medical schools [14].
While we have used consecutive versions of the ICAP for several years, we have not assessed its effectiveness or appeal to users. Therefore, we studied the following questions:
- 1.
Is the Improving Communication Assessment Program effective in training users to use the PCOF?
- a.
Is there adequate agreement between trainee and expert ratings?
- b.
Is the training easy to understand and complete?
- c.
Is there a correlation between ease of use and agreement with experts?
- a.
- 2.
Does the order of viewing the ICAP videos make a significant difference in agreement with the experts and ease of use?
- 3.
What are the most important concepts that trainees learn?
- 4.
What concepts confuse trainees?
Section snippets
Methods
ICAP programing randomized users to view either the COMMON or BETTER videos first. All responses were anonymous. Prior to viewing videos participants were asked to enter (through typing) age, gender and professional role. After rating both videos participants were asked to describe the “most important concept/skill learned” and concepts “that were confusing”, and rate the ease of using the ICAP training from 1 (extremely difficult to understand and complete) to 5 (very easy to understand and
Results
A total of 211 trainees completed the ICAP training between October 2011 and June 2012. Sixty-five percent were female. Most users (64.5%) were below 30 years of age. The majority were medical students (72%), 8% were Nurses, 4.7% Behavioral Scientists, 4.3% faculty physicians, 1.9% Medical Residents and 7.6% Other. No statistically significant differences were found for comparisons of agreement scores by gender, age (below 30 years of age vs older), and professional role (medical student vs all
Discussion
The Improving Communication Assessment Project (ICAP) (http://uwfamilymedicine.org/pcof) was effective in helping trainees learn to use the Patient Centered Observation Form (PCOF). We found strong agreement between trainee ratings and expert ratings. Users found this online training easy to understand and complete. However, we found no relationship between ease of use and agreement with experts. Training effectiveness appears stronger when rating a less sophisticated skill demonstration before
Conclusion
The Improving Communication Assessment Project helped trainees effectively and efficiently identify and name specific communication skills on the Patient Centered Observation Form. To our knowledge no other communication assessment tool has been evaluated for trainee use or widespread adoption by medical educators. Regularly using the PCOF to observe colleagues during the later years of medical school through residency and into practice may help physicians address long standing struggles to
Role of funding
None.
Conflict of interest statement
Mr. Mauksch receives fees and honoraria for providing training and consultation on communication skills to educational institutions and health care organizations.
Misbah Keen, Jan Carline and Jeanne Cawse-Lucas—None.
Acknowledgements
The authors wish to thank Valerie Ross, MS, Tom Greer, MD, MPH and Frederick Chen MD, MPH for their help creating expert ratings of the ICAP videos.
References (35)
- et al.
Faculty and medical students’ perceptions of teaching and learning about the doctor–patient relationship
Patient Educ Couns
(2010) - et al.
The four habits coding scheme: validation of an instrument to assess clinicians’ communication behavior
Patient Educ Couns
(2006) - et al.
Identifying transparency in physician communication
Patient Educ Couns
(2011) - et al.
Lexicon creation to promote faculty development in medical communication
Patient Educ Couns
(2009) - et al.
Patient-centered medicine: transforming the clinical method
Transforming the clinical method
(2003) Essential elements of communication in medical encounters: the Kalamazoo consensus statement
Acad Med
(2001)- et al.
Why the nation needs a policy push on patient-centered health care
Health Aff (Millwood)
(2010) - et al.
Integrating communication training into a required family medicine clerkship
Acad Med
(2004) Lessons learned from a peer review of bedside teaching
Acad Med
(2004)