Caveat Venditor: Throwback To The Days Of Junk Employment Statistics

June 16th, 2016 / By Kyle McEntee

Originally published on Above the Law

Welcome to the second installment of Caveat Venditor, a series that assesses claims made by law schools to separate truth from fiction. This week we look at Brooklyn Law School’s employment rate of 92.2% posted on its “By The Numbers” infographic.

I noticed this claim on Brooklyn’s website after investigating the concern of a prelaw advisor. At the quadrennial Pre-Law Advisor National Council conference, this prelaw advisor asked what to do when a law school does not meet the accreditation requirements by not publishing the required disclosures. Indeed, Brooklyn was publishing an old report nearly six months after the ABA required them to publish its new one. Brooklyn remedied this problem on Monday, citing an “oversight due to transitions in several administrative departments in the last year.” According to a spokesperson from the law school, the ABA did not follow up with the law school to make sure it published the materials on time or at all.

(The answer, by the way, is to contact the ABA Section of Legal Education, the law school, or my organization, Law School Transparency.)

After the continued bad news about the job market for law school graduates, 92.5% looks fantastic to students looking at Brooklyn Law School. It’s therefore worth examining further.

The school graduated 336 students, 300 of whom had any job — whether it requires bar passage or not, is full-time or part-time, is long-term or short-term, or is funded by the school or not. Dividing 300 into 336 produces a rate of 89.3%, short of 92.5%. That means we have to make adjustments to this equation’s numerator, denominator, or both.

In this case both.

To get to 92.5%, Brooklyn adds to the numerator three graduates who were unemployed as of the reporting deadline, but had accepted a job with a deferred start date. The school also adds another three graduates who were unemployed as of the reporting deadline, but began “full-time or part-time employment after the March 15 deadline.” (Source.)

From the denominator, Brooklyn removes two graduates who are unemployed but not seeking a job, as well as three graduates who are pursuing an additional degree. (300 + 3 + 3) / (336 – 2 – 3) = 92.5%.

The employment rate on the infographic has an asterisk on it. Follow the asterisk to the bottom of the page and there’s a link. At that link, Brooklyn refers to an “adjusted employment rate” and explains the rate in minimal detail. Even with the explanation, however, the figure is misleading.

Indeed, it’s a return to the way things were before the law school transparency movement. Law schools and the ABA maintained a tapestry of fictional statistics that deceived the public. Schools advertised employment rates north of 90 percent without disclosing that its parts were… not what consumers thought. Much of the deception was unintentional because law school administrators were afflicted by the same cultural conditioning that afflicted applicants who saw these statistics and confirmed their perceptions of law school.

But today, law school administrators know better. In some ways, that makes the efforts by Brooklyn Law School more egregious. They are aware of the history of the problem and why there was an outcry to stop.

Even with underlying data available elsewhere, the school’s infographic misleads readers and is inconsistent with industry norms. This is confirmed by the ABA’s accreditation standards and the ABA’s guidance memo on Standard 509 compliance. Standard 509(a) requires that any materials published by the law school are “complete, accurate and not misleading to a reasonable law school student or applicant.” The guidance memo reads:

Any analysis or elaboration of the data that Standard 509 mandates must be placed in the same portion(s) of the school’s website as the mandated disclosures or with prominent links to them. Further, any analysis or elaboration of the data must come after the information that is published in the form and manner designated by the Council. Finally, the display of the analysis and elaboration of the data may not be more conspicuous or prominent than the display of the mandated disclosures.

Ultimately, the problem Brooklyn faces is one of both form and of substance. The required link is not prominent, it is after the analysis/elaboration, and the glossy statistic is more prominent than the link, even with the asterisk.

There’s also the problem of the statistic being used at all. First, adding graduates who obtained jobs after the reporting deadline leads to inconsistent figures that provide a leg up for Brooklyn over its peer schools. A spokesperson for the school said the jobs were obtained “shortly after” the reporting deadline. One important question is: how far past the date is too far? The ABA employment data measure employment status as of a specific date that applies to all ABA-approved law schools. “A reasonable law student or applicant” will assume consistency among schools — rightly so. The answer, therefore, is that it does not matter how far after, the jobs should not count.

Second, the number is just not meaningful. It includes part-time jobs, short-term jobs, and jobs funded by the law school.

Fortunately, there are resources out there that help people understand Brooklyn Law School’s statistics. You can use the ABA’s website, my organization’s website, and even Brooklyn’s disclosures elsewhere (here and here) on their site. But that’s no excuse for Brooklyn Law School. Its administration should know better by now.

Today, I spoke to three administrators at the law school about these observations. One person said the following in defense:

What we’ve done historically is taken the ABA data, which we are required to report and report in ABA form, and what we do is take a slightly adjusted rate because we feel we shouldn’t be penalized for those who are not seeking [a job] or for people who have gone on to graduate degrees, that the ABA does not count as employed.

The second person said that the school is therefore “more transparent” and that “[Brooklyn Law School is] giving more information than the ABA — it gives a more fulsome picture than just the raw data. Our feeling is that we’re going above and beyond what the ABA requires.”

The third person’s comments right after really drive home the problem with this statistic. She said “it’s an overall snapshot.”

Brooklyn Law School is not alone in still using these type of figures. Its peer right across the river, Seton Hall, advertises a similar employment rate on their website.

What’s frustrating here is that Seton Hall does not even need to use such a silly statistic. Its job outcomes in 2015 were quite good because the school cut its enrollment quite far due to its commitment to maintaining its admissions standards. For the class of 2015, 79.4% of all Seton Hall graduates were in long-term, bar-passage required jobs, plus 11.8% in other professional long-term, full-time jobs. The school goes above and beyond with sound advice to applicants on its blog, and is also among the more than 60% of law schools publishing their NALP Reports each year.

In response to my request for Seton Hall to stop using this statistic, Kathleen Boozang, the school’s dean, told me the following:

Seton Hall is comfortable with and proud about the placement of all of its graduates. We work hard to help our graduates launch careers that fulfill their aspirations. I am happy to own the outcomes because it fulfills our moral responsibility to our students. We believe our prospective students will benefit by focusing more specifically on the long-term, full-time jobs that require bar passage or where the JD is an advantage.

I’m happy to see Seton Hall commit to fixing the problem.

The underlying data used by both schools highlight the problems with the statistic well. Based on the “snapshot” figures, Seton Hall and Brooklyn appear to have very similar job outcomes. But when you peel back the layers, you can see how the employment outcomes differ. After all, if a school did not think the glossy job rates discussed above would be acted upon by potential students, the school would not publish them so conspicuously.