The Latest Change in the MBE

September 5th, 2016 / By Deborah J. Merritt

In the memo announcing results from the July 2016 MBE, Erica Moeser also notified law school deans about an upcoming change in the test. For many years the 200-question exam has included 190 scored items and 10 pre-test questions. Starting in February 2017, the numbers will shift to 175 scored items and 25 pre-test ones.

Pre-testing is an important feature of standardized exams. The administrator uses pre-test answers to gauge a question’s clarity, difficulty, and usefulness for future exams. When examinees answer those questions, they improve the design of future tests.

From the test-taker’s perspective, these pre-test questions are indistinguishable from scored ones. Like other test-makers, NCBE scatters its pre-test questions throughout the exam. Examinees answer each question without knowing whether it is a “real” item that will contribute to their score or a pre-test one that will not.

So what are the implications of NCBE’s increase in the number of pre-test items? The shift is relatively large, from 10 questions (5% of the exam) to 25 (12.5% of the exam). I have three concerns about this change: fair treatment of human research subjects, reliability of the exam, and the possible impact on bar passage rates. I’ll explore the first of these concerns here and turn to the others in subsequent posts.

(more…)

» Read the full text for The Latest Change in the MBE