Supreme Court issues letter relating to In re California Bar Exam

[Text of letter below]

October 18, 2017
Michael G. Colantuono, President, Board of Trustees
Leah Wilson, Executive Director State Bar of California
180 Howard Street
San Francisco, CA 94105

Re: S244281 — In re CALIFORNIA BAR EXAM

Dear Mr. Colantuono and Ms. Wilson:

In view of the rising costs of legal education and the financial hardship potentially resulting from non-admission to the California bar, the court determined last February to assess whether the current pass score (cut score) of 1440 for the California bar exam is appropriate for evaluating the minimum competence necessary for entering attorneys to practice law in this state. Because the pass score has remained constant for three decades as overall bar exam pass rates have fluctuated, the court directed the State Bar to conduct a thorough and expedited study of the exam that would include, among other things, a meaningful analysis to determine whether protection of potential clients and the public is served by maintaining 1440 as the pass score. The court also directed that experts and stakeholders participate in the study process, including psychometricians and law school faculty or deans. The court indicated it would consider the application of any recommended adjustments to administration of the July 2017 bar exam if the recommendations are timely and found appropriate.

The current pass score was set in 1987. In the 30 years since then, the pass rate has risen and fallen over time without any change in the pass score. In contrast to the 56.1 percent pass rate in 2016 for first-time takers of the July general bar exam, there were periods of high first-time pass rates ranging from the upper 60s to the middle 70s from 1989 to 1997, 2001, and 2006 to 2013. The pass score did not appear a matter of controversy during these periods. Although the lower pass rates associated with the recent administrations of the California bar exam have generated concerns, the downward trend in pass rates appears to be consistent with a broader national pattern: statistics published by the National Conference of Bar Examiners disclose that the decline of California’s overall pass rate from 2007 to 2016 mirrors the average nine percent decline of overall pass rates observed nationally over the same period. (2016 Statistics (Mar. 2017) 86 The Bar Examiner 1, 30-33.)

The court acknowledges that California’s pass score of 1440 is the second highest in the nation and that this score was not established through a standard setting study. At the same time, there is no information indicating that many or most states selected their lower pass scores for their respective bar exams based on a standard setting analysis.

On September 13, the court received the State Bar’s “Final Report on the 2017 California Bar Exam Standard Setting Study.” The report addressed a standard setting study that was designed and conducted by independent psychometrician Chad Buckendahl, Ph.D. That study utilized a modified version of the Analytic Judgment Method, a method used in the field of licensure, and involved the participation of twenty court-selected subject matter experts (SMEs). The study required the panel of SMEs to evaluate and rate a collection of bar exam essay and performance test answers from the July 2016 exam using a baseline definition of minimum competence. After the SMEs analyzed bar exam written responses and were trained to sort them by distinguishing those reflecting at least minimal competence from those that did not reflect minimal competence, Dr. Buckendahl separately analyzed the actual scores given to such responses during the July 2016 exam scoring process to assess exam responses that were deemed minimally competent. Dr. Buckendahl then derived estimated overall bar exam scores based on the actual scores associated with written responses deemed minimally competent, and concluded that the median score associated with the SMEs’ determination of minimal competence was 1439. Opinions of the study were mixed: two independent psychometricians identified flaws in the study but ultimately found its process and conclusions sound, while a number of legal educators and others concluded the flaws of the study were so significant as to render it unreliable.

The court has reviewed the standard setting study, the State Bar’s Final Report, the amicus letters submitted pursuant to the court’s order of September 14, 2017, and the policy concerns raised in submissions on this issue. Based on that review and balancing all considerations, the court is not persuaded that the relevant information and data developed at this time weigh in favor of departing from the longstanding pass score of 1440. In making this determination, the court expects the State Bar to complete its other bar exam studies and to continue analyzing whether the exam or any of its components might warrant modification. The court also encourages the State Bar and all California law schools to work cooperatively together and with others in examining (1) whether student metrics, law school curricula and teaching techniques, and other factors might account for the recent decline in bar exam pass rates; (2) how such data might inform efforts to improve academic instruction for the benefit of law students preparing for licensure and practice; and (3) whether and to what extent changes implemented for the first time during administration of the July 2017 exam — that is, adoption of a two-day exam and equal weighting of the written and multiple choice portions of the exam — might bear on possible adjustment of the pass score. Examination of these matters could shed light on whether potential improvements in law school admission, education, and graduation standards and in State Bar testing for licensure, combined with effective regulatory oversight of legal education, could raise bar exam pass rates and thereby reduce financial hardship for exam takers, and boost the availability of competent and effective attorneys across all demographics and for all Californians.

Although the current pass score of 1440 will not be adjusted at this time, the court will consider any appropriate recommendation to revisit the pass score in the next review cycle, or sooner if the court so directs. (See Cal. Rules of Ct., rule 9.6(b), eff. Jan. 1, 2018.)

Sincerely,

Hon. Tani G. Cantil-Sakauye
Chief Justice

Hon. Ming W. Chin
Associate Justice

Hon. Carol A. Corrigan
Associate Justice

Hon. Goodwin Liu
Associate Justice

Hon. Mariano-Florentino Cuéllar
Associate Justice

Hon. Leondra R. Kruger
Associate Justice