Eva is an Associate Teaching Professor at the University of Notre Dame, where she is also a fellow of the Kellogg Institute for International Studies, the Liu Institute for Asia and Asian Studies, and the Pulte Institute for Global Development. She received her PhD from the University of Illinois, Chicago in 2014.
Last June, we interviewed Eva on our podcast. That podcast can be found HERE.
Can a Behavioral Nudge with Small Commitment Lead to Better Exam Scores?
So Covid brought challenges…. we can’t really even count them. In the world of education, it meant switching to online delivery and while that may be hard on us professors, it also requires a lot more from students. Learning from home requires more discipline, there is a degree of freedom (statistics pun intended). There is also a lack of accountability that typically comes with attending an in-person class where the professor can call you out for not being prepared. This is what non-traditional students who have a job, a family, go through on a regular basis even without Covid. The often opt for online classes in the first place. It can also be a tougher adjustment for students who come from traditionally underrepresented groups in higher education, as they may not grow up watching their parents make lists, prioritize, and manage time that would promote college success. Is there something that could help alter students’ behavior and overcome this inequality?
In all of our introductory economic models, we assume that agents are rational. If that assumption is violated, we cannot really predict how they will respond to incentives and our models would lose their predictive power. The 2017 Nobel Prize in Economics was awarded to Richard Thaler for his contributions to behavioral economics. The art and science of “irrationality”. Well, about time as we seem to violate rationality a lot! We know we should study, we know we should not procrastinate, yes we know but… These choices can have serious long-term consequences, so it is important to study our behaviors and why we make decisions that perhaps do not appear rational. And it is important to study how we could alter certain behaviors. Research has shown that simply nudging students with a text message doesn’t really lead to improvements in academic performance. A 2019 NBER working paper summarized it pretty well: “The Remarkable Unresponsiveness of College Students to Nudging and What We Can Learn from It.” [The paper can be found HERE.]
In our paper “Microcommitments: The Effect of Small Commitments on Academic Performance,” we set out to test whether a text message nudge accompanied by a small commitment can “push” students in the right direction. In economics, the gold standard of answering questions like this is a randomized controlled trial. If assignment is not random and students are selected into treatment and control groups, then we would not be able to identify the role of the intervention, as these groups may be responding differently in the first place. For example, imagine we tested the nudge with commitment on a group of women and men served as the “placebo” control group. If we find higher exam scores for women, then it may be because of the nudge with commitment but it is also entirely possible that women could have scored higher regardless, this is referred to as a selection bias. We overcome this by randomly assigning almost 1,000 students from the University of Notre Dame, Florida Atlantic University, and University of Illinois into two groups, which after close examination of observable characteristics look very similar. This is called a balance test. After randomization, the two groups have a similar proportion of women, similar average SAT, GPA, age, family structure, their procrastination tendencies, self-efficacy, study habits, etc. Some of the students are enrolled in regular in-person classes, and some are enrolled in a hybrid/online classes.
After the first exam of the semester, which will serve as a baseline comparison, the experiment begins! Both groups receive text messages in the morning with content related to material covered in class. Students know they are not required to submit their answers and it is not mandatory, these messages are just extra practice on how to think as an economist. The control group received the content as a simple text message. The treatment group’s text message also had “I commit” to click. Then at 4pm, they also got a follow up text with “I did it” click. This is the commitment device we are testing and it is the only difference between the treatment and control groups, everything else is identical. The research question is: Does a small commitment (really to yourself, as it is not required) compel you to complete the task and does this engagement then improve your future exam score? The regression estimation allows us to hold everything else constant, so we are adhering to our ceteris paribus condition.
It turns out that the small commitment does make a difference! In fact, the positive results on the exam which followed the experiment is driven by students in hybrid and online classes who scored 3.5 percentage points higher than students in the control group which received the same message content but did not receive the commitment! We find no effect on the academic performance among students in regular in-person classes. It appears that this simple intervention partially substitutes for the lack of instructor contact for students in hybrid and online classes. We also find that students who tend to procrastinate and those with lower GPA benefit from the commitment device more, which then acts as an equalizing force in terms of academic performance and could have positive implications for social mobility and economic equity. Who would have thought that making a small promise to yourself could actually make a difference!!!
References: Felkey, Amanda J, Eva Dziadula, Eric P Chiang, and Jose Vazquez. 2021. “Microcommitments: The Effect of Small Commitments on Academic Performance.” AEA Papers and Proceedings 111: 1–6. [The paper can be found HERE.]
Oreopoulos, Philip, and Uros Petronijevic. 2019. “The Remarkable Unresponsiveness of College Students to Nudging and What We Can Learn from It.” [The paper can be found HERE.]
HubbardOBrienEconomics.com Blog 