By Debra Werrlein
First Prize Poster: “Building an Innovative, Competency-Based Assessment System to Support a Renewed, Small Group Learning-Based, Spiraled Dental Curriculum at the University of British Columbia”
What if students wrote their own exam questions and the exams weren’t graded?
For students at the
University of British Columbia Faculty of Dentistry (UBC Dentistry), these ideas aren’t as farfetched as they sound. At the 2016 ADEA CCI Liaisons Summer Meeting in June, faculty from UBC Dentistry presented the new assessment system they’ve designed to support their recently revised curriculum.
UBC Dentistry’s basic science curriculum, like those at many dental schools around the world, was integrated with that of the Faculty of Medicine. When the latter set out to revamp its curriculum, UBC Dentistry took the opportunity to look at whether the
proposed changes would meet the needs of dental students. They already knew that portions of the basic science curriculum were not directly relevant to dental students, but the problem proved worse than originally thought. For example, reviewers discovered that only five percent of the information taught about the
reproductive system was relevant for dentists. An immediate change was made, but broader revisions were clearly in order. UBC Dentistry decided to split from the Faculty of Medicine to allow dental educators to control the information delivered to students, and when and how to deliver it.
The First Step
To renovate the curriculum for UBC Dentistry, working groups surveyed 1,500 existing biomedical science objectives to create a new set of “need to know” objectives that directly relate to dentistry. Working groups also revised or removed medical cases as needed,
determined the depth of content to be included, and created a competency framework that aligns with the dental health competencies put forth by the Association of Canadian Faculties of Dentistry.
In addition to this new control over what material students learn, UBC Dentistry can now decide when and how they learn it. This freedom enables faculty to spiral the introduction of content into the curriculum and to move away from the previous 2 + 2 format, which called
for students to learn basic sciences in the first two years and engage in clinical work during the second two years. First-year students are now benefitting from the kind of hands-on experience they asked for—thanks to UBC Dentistry faculty’s having integrated a restorative dentistry simulation into the program’s first
With a new, more dentally relevant curriculum in the works, faculty at UBC Dentistry understood they would need an assessment system that reflected the changes they were making. Recognizing this need led to creating UBC Dentistry’s “Progress Portfolio,” an innovation that includes two primary elements: a learning tool
called PeerWise and a semi-annual assessment tool called the Progress Survey. According to W. Leandra Best, D.M.D., Clinical Professor and Associate Dean of Academic Affairs at UBC Dentistry, the Progress Portfolio has two aims: to support the new curriculum and contribute to learning, rather than just measure it.
PeerWise is a free, online learning tool that allows students to create, answer and comment on a pool of multiple-choice questions. “This has great educational value,” says Dr. Best. “Students learn by formulating questions and by rationalizing the answers they
create.” Not only must students write an accurate correct answer, they must also understand the material well enough to write answers that are realistic but wrong.
UBC Dentistry’s 2015–16 entering students were the first entire class to participate in PeerWise, and the positive response was overwhelming. Instead of grading the questions students created, faculty selected the highest-quality questions and included them in formal
assessments. Dr. Best says the prospect of seeing their own questions on an exam was a carrot for students, who surprised faculty with both their enthusiasm and the high quality of the questions they wrote.
After seeing such positive results, UBC Dentistry recognized the potential to create a bank of exam questions. The excitement extended beyond the faculty members. Students were so inspired by the way PeerWise involved them in their own education, a group volunteered
to work with faculty over the summer to further refine assessment questions and case studies. Faculty from other universities—some of whom learned about UBC Dentistry’s PeerWise project at the 2015 ADEA Annual Session & Exhibition—have recognized the potential in using this free, online tool and have expressed an
interest in partnering to create an even bigger pool of exam questions.
The second component of UBC Dentistry’s assessment innovation is the Progress Survey. The survey comprises faculty-generated questions that align with the new curriculum combined with student-generated questions developed through PeerWise. Faculty administer this three-hour,
200-question, multiple-choice test twice a year to assess student progress toward achieving competency in all knowledge domains. This testing schedule means that beginning in their first year, students are tested on all of the material they will learn in dental school.
Implementation of the Progress Survey might sound like a crazy idea—indeed, attendees at this summer’s poster presentation raised questions about students possibly becoming demoralized by how much they do not know—but Dr. Best says students found it
motivating. The survey is formative, so in place of a grade, UBC Dentistry produces individual “personal progress index maps” to illustrate outcomes. These maps show students’ progress toward achieving competency in each dental division, giving them a visual representation of where they are in the four-year program.
“We’re trying to give lower-stakes assessments that aren’t just about the mark,” says Dr. Best, “because we believe students progress more quickly this way. Students are more comfortable asking questions and talking to professors when they know their
questions won’t count against them.”
The test, however, should do more than change student attitudes. “The idea is for the test to be a learning experience in and of itself,” says Dr. Best. She explains that the repetition of material can increase retention while the exposure can help students see the relevance of what they’re learning and understand the
expectations they must meet to progress through the program.
UBC Dentistry will continue its transition to the new curriculum and corresponding assessment system over the next three years as the first-year class moves toward graduation. Expectations are high. “We’ve tried to enrich the educational experience for dental students so they feel important and enthused and
understand why they’re learning what they’re learning,” says Dr. Best, who hopes to see a continuation of the high levels of enthusiasm demonstrated by students so far.
Dr. Best and her UBC Dentistry colleagues Nancy Black, M.Sc., D.M.D., Clinical Associate Professor; HsingChi von Bergmann, M.Sc., Ph.D., Associate Professor; and James Richardson, D.M.D., M.E.T., Clinical Associate Professor, co-authored the poster submission
that won first prize at the 2016 ADEA CCI Liaisons Summer Meeting. They were all thrilled to see a Canadian school win and plan to use the prize money to further their curriculum renewal projects.