Does Feedback Improve Performance?

April 27th, 2017 / By Deborah J. Merritt

My colleague Ruth Colker gives her 1L students the opportunity to obtain mid-semester feedback on their written work. In her Constitutional Law course, currently taught during the spring semester of the first year, Ruth invites students to submit a practice answer to an essay question drawn from a previous exam. She grades each practice answer using the same rubric she used on the previous final, makes extensive written comments on the answers, and encourages students to discuss their answers with her in person.

The exercise is not mandatory; nor does it factor into the final grade. About half of Ruth’s students choose to obtain this optional mid-semester feedback. She wondered if those students performed better on the final exam than students who did not elect the feedback. To study this question, she assembled a team of colleagues, including an expert statistician, Abigail Shoben, from Ohio State’s College of Public Health.

I was delighted to work as part of this team, which also included law colleagues Ellen Deason and Monte Smith. We’ve just published our results. Here are some of the highlights:

Feedback Was Associated with Better Performance on the Class Final

We studied three consecutive years of Ruth’s students–167 students in  all. Eighty-one of those students (49%) completed the practice exam question. Those students, we found, scored significantly higher on Ruth’s final exam than classmates who did not obtain Ruth’s formative feedback. This difference persisted after controlling for LSAT score, UGPA, race/ethnicity, sex, and fall-semester law grades. After controlling for the latter factors in a multiple regression equation, students who took the practice exam scored an average of 2.3 points (about half a letter grade) higher on the final exam than their classmates did.

Those 2.3 points, of course, reflect an association rather than causation; the analysis does not prove that formative feedback caused  better performance on the final exam. Our natural experiment, however, controlled for many other factors that might have affected performance. All of the students had the same Constitutional  Law professor and an identical schedule of other courses. (We excluded from our study students who light-loaded.) Within each year, the students all took the same final exam in Constitutional Law and had the same professors/workloads in other courses. We also controlled for factors like LSAT and UGPA that often correlate with law school performance.

Perhaps most important, we controlled for the grades that the students received during their first law-school semester. If the students who took the optional practice exam were particularly conscientious students–ones who would have scored well on the final exam even without the practice opportunity–that conscientiousness should have been reflected in their fall-semester grades. Even after controlling for prior law school performance, however, formative feedback was associated with better course outcomes.

Feedback Was Associated with Better Performance in Other Classes

Even more intriguing, we found that the students who completed Ruth’s practice exam obtained higher grades, on average, in their other spring-semester courses. Once again, this relationship persisted after controlling for the factors listed above. The association was both statistically and practically significant. In terms of class rank, completing the practice exam was associated with moving up from the fiftieth percentile to the fifty-ninth–based purely on the improved grades in courses other than Constitutional Law. Combining the bump in Constitutional Law grades with that in other courses yielded even greater gains.

As above, this analysis shows a relationship; it does not prove causation. Daniel Schwarcz and Dion Farganis, however, recently showed a similar relationship for first-year students who obtained feedback at the University of Minnesota Law School. Andrea Curcio and her colleagues have also published studies documenting a relationship between formative assessment and law student achievement. Taken together, these studies should at least provoke serious discussion about the benefits of adding formative feedback to the law school curriculum.

Gender and Feedback

The association between feedback and better grades occurred for both male and female students in our study; gender did not affect that relationship. Gender, however, was significant in predicting who would choose to take the practice exam: women were significantly more likely than men to seek Ruth’s formative feedback.

This gender relationship could stem from Ruth’s own gender. Perhaps female students were particularly eager to interact with her and seek her feedback. The relationship might also reflect women’s persistent unease about their place in law school. Although women now make up just over half of all law students, legal education still displays some features of a male-dominated profession. As the “newcomers” to this profession, women might seek more reassurance and feedback. Alternatively, female students in general may be more willing to seek formative feedback than their male counterparts.

Whatever the explanation for this gendered relationship, it reminds us that pedagogic decisions are rarely neutral. Some approaches may encourage one group of students, while a different approach encourages others. Our traditional law school pedagogy, which offers very little formative feedback, may be yet another vestige of a male-dominated profession. The good news is that adding feedback to the curriculum seems to benefit male and female students equally–once they obtain that feedback.

Race/Ethnicity and Feedback

Our study identified no significant relationship between race/ethnicity and the outcomes we studied. Nonwhite students were no more or less likely to take the practice exam question than white students. Nor did race/ethnicity affect the relationships we identified between taking that practice exam and course outcomes. We did observe one intriguing relationship: nonwhite students with LSAT scores at or above our College median performed particularly well in Ruth’s course. Perhaps there are other elements of Ruth’s course that helped those students succeed; we will try to follow that trail in further research.

Conclusion

We still have much to learn about the relationship between formative feedback and law student success. Our study examined just one type of feedback, offered by one professor in a particular course. Other studies in this area also suffer from methodological limits. The findings are sufficiently compelling, however, to promote more research, discussion, and experimentation in this area. We can no longer assume that our traditional no-feedback pedagogies produce the best outcomes for students. For more details of our study, you read the full paper here.