Measuring clinical skills in agenda-mapping (EAGL-I)

https://doi.org/10.1016/j.pec.2015.06.018Get rights and content

Highlights

  • We developed EAGL-I, a measure of agenda-mapping for use in teaching and research.

  • We demonstrate reliability and validity of EAGL-I scores.

  • Higher EAGL-I scores predict greater likelihood that a patient will disclose their full agenda in a simulated environment.

Abstract

Objective

To develop and validate the Evaluation of AGenda-mapping skilL Instrument (EAGL-I).

Methods

EAGL-I was constructed after a literature review and piloting. Simulated consultation recordings were collected in a workshop with third-year medical students at three time points: once pre-teaching, twice post-teaching. Three raters used EAGL-I to assess student agenda-mapping. We examined reliability, ability to detect change and predict full expression of patients’ agendas.

Results

EAGL-I scores represented reliable assessment of agenda-mapping (Ep2 = 0.832; φ = 0.675). Generalizability coefficients across items (Ep2 = 0.836) and raters (φ = 0.797 two raters) were acceptable. A one-way repeated measure ANOVA with post hoc analysis found a statistically significant difference between the pre-teaching occasion of measurement and each post-teaching occasion (p < 0.001) and no significant difference between the two post-teaching occasions (p = 0.085). Multilevel logistic regression show scores predict expression of scripted hidden agendas irrespective of occasions, or patient scenario (n = 60, p = 0.005).

Conclusion

Evidence of measure validation is shown. Reliability is optimised when two or more raters use EAGL-I and agenda-mapping has been taught. EAGL-I appears sensitive to change. Higher scores predict the likelihood that a patient will disclose their full agenda in a simulated environment.

Practice implications

A validated tool for measuring agenda-mapping in teaching and research is now available.

Introduction

In clinical consultations there are often a range of inter-related and sometimes competing priorities to consider. Practitioners and patients may well have different views about which is the most pressing priority for the consultation and for promoting short- and long-term health and well-being. The call to involve patients meaningfully in their care has been well made [1], [2], [3]. Consequently it is important for both patients and clinicians to agree what should be discussed in a consultation.

Agenda-mapping is a collaborative communication strategy that enables clinicians and patients to establish shared focus on what to talk about by mapping options and agreeing priorities [4]. With roots in the patient-centred approach, it is a refinement and integration of well developed techniques used to establish the focus or reason for the clinical consultation [5], [6], [7], [8], [9]. In some clinical consultations, e.g. patient initiated primary care visits, eliciting the full patient agenda might be the main emphasis of these techniques [5], [6], [7], [8]. However even in these consultations, clinicians may have agendas they may wish to raise opportunistically [10]. In other consultations e.g. routine review of long-term conditions, clinicians may have well formulated agendas, some aspects of which may involve behaviour change [9], [11]. In this instance a clinician might choose to use a particular approach, such as Motivational Interviewing (MI), to further discuss the topic raised [12]. However agenda-mapping is essentially a strategy to help establish focus, which is a generic communication skill.

Simply describing a strategy like agenda-mapping might not be sufficient to understand its use or to integrate it into the teaching of practitioners. It is important to both clearly define and to measure it. From there we can consider whether practitioners can acquire these skills, and what effect agenda-mapping might have on consultation processes and outcomes.

Measurement is an essential part of good clinical practice and research in the health sciences [13], [14]. Well-developed measures can result in better decisions being taken about individuals and programs in education, research and healthcare delivery [13]. In research settings measures are used to assess outcomes or to support process analyses and fidelity checks [15]. In educational settings, they can facilitate both the development and assessment of competence [15]. The use of checklists and measures to assess clearly defined elements of communication is well developed in teaching settings. To be useful, approaches to measurement should be both psychometrically sound and clinically beneficial [14] and measure development is therefore shaped by consideration of its purpose, format, and both the context and consequence of its use [13].

We aimed to develop a measure of agenda-mapping that could be used in teaching. We first considered the conceptual foundation for this work by clarifying the definition and core domains of agenda-mapping. This work is reported elsewhere [4]. Next we considered if and how others have measured these domains. We then developed a measure, the Evaluation of AGenda-mapping skilL Instrument, and tested it in a clearly defined context: primary care, in undergraduate medical education (third year medical students).

Section snippets

Review of existing measures

We reviewed the healthcare literature to identify if and how agenda-mapping domains have been included in existing communication measures. In this way we could establish the rationale for developing a new measure and also use existing measures to inform the development and design of our own. As our overarching aim was to develop a measure for teaching students and/ or clinicians, we focused our search here on measures that assess communication competence.

The following search was run on Medline,

Results

Decisions about the content, format and scoring of items were made in the development of EAGL-I, together with coding rules. This process determined three key aspects of the measure design. First, the process of coding agenda-mapping interactions, where agendas were categorized and coupled with the clinician’s response, was dropped. Raters considered it not feasible for single pass coding. Second, behavioural anchors were embedded in the score sheet to help with decision-making. And third, a

Discussion

We developed a measure that could be used to assess agenda-mapping in a teaching context. The measure’s content was developed from conceptual development work [4] and a review of existing measures. Results from a study with third year medical students suggest that when two or more raters’ observations are averaged, EAGL-I scores provide a highly reliable assessment of agenda-mapping. The measure detects change in agenda-mapping after teaching and higher EAGL-I scores are related to a greater

Conclusion

We have developed EAGL-I, a measure of agenda-mapping and have demonstrated that EAGL-I scores represent reliable and valid assessment of student agenda-mapping. Reliability is optimised under the following conditions: if a student has been formally taught agenda-mapping, if a student has been observed on 2 or more occasions and their scores are compared (e.g. by looking at a pre-teaching occasion and a post teaching occasion), and if two or more trained raters are used for each observation.

Author contributions

NG—study lead, concept, design, conduct, analysis, write up; PK, JG, MR—supervised study, concept, design, analysis, write up, KH, TP—analysis and write up. All authors reviewed and contributed to manuscript write up.

Acknowledgements

Stephen Rollnick co-delivered the teaching of third year medical students. Claire Lane, Ian Cooper and Phillippa Thomas, rated the audios. Ralph Bloch and Geoff Norman offered expert guidance in the application of Generalizability theory.

This work was funded by Cardiff University.

References (65)

  • R.L. Street et al.

    How does communication heal? Pathways linking clinician–patient communication to health outcomes

    Patient Educ. Couns.

    (2009)
  • T. Bodenheimer et al.

    Patient self-management of chronic disease in primary care

    J. Am. Med. Assoc.

    (2002)
  • Department of Health

    Liberating The NHS: No Decision About Me, Without Me—Government Response to the Consultation

    (2012)
  • J. Greene et al.

    Why does patient activation matter? An examination of the relationships between patient activation and health-related outcomes

    J. Gen. Intern. Med.

    (2012)
  • H.B. Beckman et al.

    The effect of physician behavior on the collection of data

    Ann. Intern. Med.

    (1984)
  • L. Dyche et al.

    The effect of physician solicitation approaches on ability to identify patient concerns

    J. Gen. Intern. Med.

    (2005)
  • M.K. Marvel et al.

    Soliciting the patient’s agenda: have we improved?

    J. Am. Med. Assoc.

    (1999)
  • L.B. Mauksch et al.

    The establishing focus protocol: training for collaborative agenda setting and time management in the medical interview

    Fam. Syst. Health

    (2001)
  • N.C.H. Stott et al.

    Innovation in clinical method, diabetes care and negotiating skills

    Fam. Pract. Manag.

    (1995)
  • N.C.H. Stott et al.

    The exceptional potential in each primary care consultation

    J. R. Coll. Gen. Pract.

    (1979)
  • M. Robling et al.

    The effect of the Talking Diabetes consulting skills intervention on glycaemic control and quality of life in children with type 1 diabetes: cluster randomised controlled trial (DEPICTED study)

    Br. Med. J.

    (2012)
  • W.R. Miller et al.

    Motivational Interviewing—Helping People Change

    (2012)
  • American Educational Research Association (AERA) et al.

    Standards for Educational and Psychological Testing

    (1999)
  • D.L. Streiner et al.

    Health Measurement Scales: A Practical Guide to Their Development and Use

    (2003)
  • T.R. Egnew et al.

    Integrating training into a required family medicine clerkship

    Acad. Med.

    (2004)
  • R.J. Howells et al.

    Assessment of doctors’ consultation skills in the paediatric setting: the Paediatric Consultation Assessment Tool. [Reprint in Postgrad Med J. 2010 Oct;86(1020):584-90; PMID. 20971710]

    Arch. Dis. Child.

    (2010)
  • S. Kurtz et al.

    Marrying content and process in clinical method teaching: enhancing the Calgary—Cambridge Guides

    Acad. Med.

    (2003)
  • R.C. Fraser et al.

    Consultation competence in general practice: testing the reliability of the Leicester assessment package

    Br. J. Gen. Pract.

    (1994)
  • A. Kalet et al.

    Teaching communication in clinical clerkships: models from the Macy initiative in health communications

    Acad. Med.

    (2004)
  • G. Bonner et al.

    The validation of a rating scale to assess dietitians’ use of behaviour change skills

    J. Hum. Nutr. Diet

    (2008)
  • I. Enzer et al.

    A reliability study of an instrument for measuring general practitioner consultation skills: the LIV-MAAS scale

    Int. J. Qual. Health Care

    (2000)
  • M.R. Utting et al.

    Consultation skills of medical students before and after changes in curriculum

    J. R. Soc. Med.

    (2000)
  • Cited by (4)

    • Agenda-setting revisited: When and how do primary-care physicians solicit patients' additional concerns?

      2016, Patient Education and Counseling
      Citation Excerpt :

      This type of agenda pursuit has its greatest effects when performed ‘early’ or ‘up front’ during visits (i.e., no later than history taking, and preferably immediately after patients’ presentations of their chief concerns) [15,21]. As a component of patient-centered communication [15,23–25], and ultimately of shared decision making [26], agenda pursuit is now a staple of textbooks on medical interviewing [27–29] and is a frequent component of physician-training programs [5,18,22,30–35]. It has also been integral to innovations in medical interviewing, such as the ‘four-habits model’ [26,36] and the ‘establishing-focus protocol’ [16,21,37].

    View full text