Spring 2012: Interprofessional Spotlight

The Physical Therapy Clinical Performance Instrument
By Nicole Fauteux

If formulating a way to evaluate overall student competency at a single school seems like a monumental task, imagine designing a tool that any school could use, one that would incorporate the profession's essential elements and be valid and easy to implement in clinical settings. That's what our colleagues in physical therapy have done, and more than a decade after their Clinical Performance Instruments (CPIs) were first piloted, these tools have achieved wide acceptance as a means of evaluating students' clinical performance and readiness for entry-level practice.

How our physical therapy colleagues achieved this feat may be instructive as we seek to develop similar instruments of our own. Their story begins in the early 1990s, when clinical educators were expressing frustration at having to spend large amounts of time learning and implementing different assessments for each program whose students they supervised. The idea of standardizing student assessment across programs emerged as a promising way to reduce the time preceptors spent on assessment while allowing them to do the job well and continue working with multiple physical therapist (PT) and physical therapist assistant (PTA) programs.

In 1993, the American Physical Therapy Association (APTA) took on the challenge of developing two clinical assessment tools, one for PT and another for PTA programs. According to a 2002 article in Physical Therapy, the task force began its work by agreeing on three fundamental assumptions:

(1) that clinical competence is based on multiple behaviors deemed essential to the role of the PT or PTA, (2) that the CPIs should be constructed to measure performance along a continuum from novice to at least entry level, and (3) that the instruments must be responsive to the needs of both academic and clinical communities.

The task force spent three and a half years researching and developing the CPIs. Another decade of testing and refinement followed, and today these web-based global assessment tools are widely used to evaluate students' clinical performance and readiness for practice.

"Those first years were labor intensive," says Dr. Jody Frost, PT, DPT, Ph.D., Director, Department of Academic/Clinical Education Affairs, American Physical Therapy Association, who coordinated the CPIs' development. "We had to place ourselves in the shoes of those who would use the CPI," Frost recalls. "The clinician says, don't make this really long. I don't have time. Make this clear and easy to complete. The program says, make this psychometrically sound so I can defend a grievance, provide an early warning system to ensure that due process is followed, and design an instrument that enables the program to make objective decisions about student progression and program completion related to clinical practice. The student says I want something that shows small progress so I feel like I'm getting somewhere."

Making all of those constituencies happy required patience and a willingness to adhere to a stringent process that involved multiple rounds of review and lots of listening.

"Some of our discussions in the open forum were passionately charged. I can remember a few occasions with lightning bolts," Frost jokes. "We needed to learn to listen, acknowledge, seek clarification, but not to defend. It takes a willingness to grow and mature as a group to figure out how to manage that role."

The resulting tool focuses on what the student is able to do in the aggregate, or what dental educators would call global or overall competency. Preceptors are asked to rate students on 18 performance criteria, each of which includes a set of illustrative sample behaviors that demonstrate essential or core performance outcomes expected of the well-prepared clinician. For example, the PT CPI includes the following performance criterion: "Practices in a safe manner that minimizes the risk to patient, self, and others." Behaviors that illustrate this include demonstrating knowledge of facility safety policies and procedures, recognizing physiological and psychological changes in patients and adjusting patient intervention accordingly, requesting assistance when necessary, and using acceptable techniques for safe handling of patients.

Students are rated on a categorical scale that ranges from beginning performance to beyond entry-level performance with clearly defined performance dimensions at each point along the continuum. Preceptors are asked to evaluate how well, how consistently, and how efficiently students perform, how much supervision or guidance is needed, and the complexity of the task, the patient, and the environment. Students must meet precise benchmarks in each of these dimensions in order to progress along the continuum. An article describing the latest version of the CPIs in greater detail will be published this month in Physical Therapy.

A one-size-fits-all approach to assessing overall competency in the dental professions may not be in the cards. That said, it's interesting to note that the instrument APTA developed seems to work for the vast majority of its constituents. Use of the tool is voluntary, and an impressive 187 out of 213 accredited physical therapist programs have adopted it rather than retaining or creating clinical assessment tools of their own. Even concerns that the use of a uniform tool might reveal deficiencies in a program have evolved into a commitment to quality improvement.

"All the programs want to know, how do my graduates compare to others," Frost reports. "When they've identified places where they need improvement, they've stepped up to enhance their curricula."

This makes sense given the realization among PT programs back in the 1990s that they were spending lots of time developing, revising, and updating tools without any assurance that they were reliable or valid. According to Frost, program leaders finally concluded that this was not the best use of faculty time. Today, programs that adopt the PT or the PTA CPI use it for all of their students' clinical experiences. Programs may also supplement the CPI with additional tools and requirements specific to the needs of a program and its mission. To ensure the tool's reliability, preceptors, students, and faculty who use it must pass an online training course.

This coming year, APTA will release the last component of the web-based instrument (PT CPI Web): a mechanism that enables the program to compile up to seven years of data gleaned from the PT CPI assessment into a set of pre-designed reports relevant to the program, curriculum assessment, and the accreditation process.