Guest Post from Eva Dziadula of Notre Dame on Using Behavioral Economics to Improve Test Scores

Eva is an Associate Teaching Professor at the University of Notre Dame, where she is also a fellow of the Kellogg Institute for International Studies, the Liu Institute for Asia and Asian Studies, and the Pulte Institute for Global Development.  She received her PhD from the University of Illinois, Chicago in 2014.

Last June, we interviewed Eva on our podcast. That podcast can be found HERE.

Can a Behavioral Nudge with Small Commitment Lead to Better Exam Scores?

So Covid brought challenges…. we can’t really even count them. In the world of education, it meant switching to online delivery and while that may be hard on us professors, it also requires a lot more from students. Learning from home requires more discipline, there is a degree of freedom (statistics pun intended). There is also a lack of accountability that typically comes with attending an in-person class where the professor can call you out for not being prepared. This is what non-traditional students who have a job, a family, go through on a regular basis even without Covid. The often opt for online classes in the first place. It can also be a tougher adjustment for students who come from traditionally underrepresented groups in higher education, as they may not grow up watching their parents make lists, prioritize, and manage time that would promote college success. Is there something that could help alter students’ behavior and overcome this inequality?

In all of our introductory economic models, we assume that agents are rational. If that assumption is violated, we cannot really predict how they will respond to incentives and our models would lose their predictive power. The 2017 Nobel Prize in Economics was awarded to Richard Thaler for his contributions to behavioral economics. The art and science of “irrationality”. Well, about time as we seem to violate rationality a lot! We know we should study, we know we should not procrastinate, yes we know but… These choices can have serious long-term consequences, so it is important to study our behaviors and why we make decisions that perhaps do not appear rational. And it is important to study how we could alter certain behaviors. Research has shown that simply nudging students with a text message doesn’t really lead to improvements in academic performance. A 2019 NBER working paper summarized it pretty well: “The Remarkable Unresponsiveness of College Students to Nudging and What We Can Learn from It.” [The paper can be found HERE.] 

In our paper “Microcommitments: The Effect of Small Commitments on Academic Performance,” we set out to test whether a text message nudge accompanied by a small commitment can “push” students in the right direction. In economics, the gold standard of answering questions like this is a randomized controlled trial. If assignment is not random and students are selected into treatment and control groups, then we would not be able to identify the role of the intervention, as these groups may be responding differently in the first place. For example, imagine we tested the nudge with commitment on a group of women and men served as the “placebo” control group. If we find higher exam scores for women, then it may be because of the nudge with commitment but it is also entirely possible that women could have scored higher regardless, this is referred to as a selection bias. We overcome this by randomly assigning almost 1,000 students from the University of Notre Dame, Florida Atlantic University, and University of Illinois into two groups, which after close examination of observable characteristics look very similar. This is called a balance test. After randomization, the two groups have a similar proportion of women, similar average SAT, GPA, age, family structure, their procrastination tendencies, self-efficacy, study habits, etc. Some of the students are enrolled in regular in-person classes, and some are enrolled in a hybrid/online classes. 

After the first exam of the semester, which will serve as a baseline comparison, the experiment begins! Both groups receive text messages in the morning with content related to material covered in class. Students know they are not required to submit their answers and it is not mandatory, these messages are just extra practice on how to think as an economist. The control group received the content as a simple text message. The treatment group’s text message also had “I commit” to click. Then at 4pm, they also got a follow up text with “I did it” click. This is the commitment device we are testing and it is the only difference between the treatment and control groups, everything else is identical. The research question is: Does a small commitment (really to yourself, as it is not required) compel you to complete the task and does this engagement then improve your future exam score? The regression estimation allows us to hold everything else constant, so we are adhering to our ceteris paribus condition.

It turns out that the small commitment does make a difference! In fact, the positive results on the exam which followed the experiment is driven by students in hybrid and online classes who scored 3.5 percentage points higher than students in the control group which received the same message content but did not receive the commitment! We find no effect on the academic performance among students in regular in-person classes. It appears that this simple intervention partially substitutes for the lack of instructor contact for students in hybrid and online classes. We also find that students who tend to procrastinate and those with lower GPA benefit from the commitment device more, which then acts as an equalizing force in terms of academic performance and could have positive implications for social mobility and economic equity. Who would have thought that making a small promise to yourself could actually make a difference!!!

References: Felkey, Amanda J, Eva Dziadula, Eric P Chiang, and Jose Vazquez. 2021. “Microcommitments: The Effect of Small Commitments on Academic Performance.” AEA Papers and Proceedings 111: 1–6. [The paper can be found HERE.]

Oreopoulos, Philip, and Uros Petronijevic. 2019. “The Remarkable Unresponsiveness of College Students to Nudging and What We Can Learn from It.” [The paper can be found HERE.]

More Than You Probably Want to Know about the Debate Over Whether Giving Presents Causes a Deadweight Loss

If your sister gives you a sweater that you don’t like, the subjective value you place on the sweater will probably be less than the price your sister paid for it. As we saw in Chapter 4, Sections 4.1 and 4.2, consumer surplus is the difference between the highest price a consumer is willing to pay for a good (which equals the consumer’s marginal benefit from the good) and the actual price the consumer pays. We expect that you will only buy things that have a marginal benefit to you greater than (or, at worst, equal to) the price you paid. Therefore, everything you buy provides you with positive (or, again at worst, zero) consumer surplus. But if the price is greater than the marginal benefit—as is the case with the sweater your sister gave you—consumer surplus is negative and there is a deadweight loss.

In the early 1990s, Joel Waldfogel, currently at the University of Minnesota, published an article in the American Economic Review in which he reported surveying his undergraduate students, asking them to: (1) list every gift they had received for Christmas, (2) estimate the retail price of each gift, and (3) state how much they would have been willing to pay for each gift. Waldfogel’s students estimated that their families and friends had paid $438 on average on the students’ gifts. The students themselves, however, would have been willing to pay only $313 for the goods they received as gifts—so, on average, each student’s gifts caused a deadweight loss of $313 – $438 = –$125. If the deadweight losses experienced by Waldfogel’s students were extrapolated to the whole population, the total deadweight loss of Christmas gift giving could be as much as $23 billion (adjusting the value in Waldfogel’s article to 2020 prices).

If the gifts had been cash, the people receiving the gifts would not have been constrained by the gift givers’ choices, and there would have been no deadweight loss. If your sister had given you cash instead of that sweater you didn’t like, you could have bought whatever you wanted and received positive consumer surplus.

Waldfogel’s article attracted much more attention than is received by the typical academic journal article, being covered in newspaper articles and on television. It also set off a lively debate among economists over whether his approach was valid. Waldfogel later published a short book, Scroogenomics, in which he argued that, in fact, his journal article had underestimated the deadweight loss of gift giving because it compared the value of the gift to the person receiving it to the value of receiving cash instead. He noted that a more accurate comparison would be not to cash but to the value of the good the person receiving the gift would have bought with the cash. Because that purchase would provide positive consumer surplus to the buyer, the buyer’s loss from receiving a gift rather than cash is significantly greater than he had originally calculated.

Waldfogel again surveyed his undergraduate students asking them to make this revised comparison and found (p. 35): “Dollars on gifts for you produce 18 percent less satisfaction, per dollar, than dollars you spend on yourself.” Waldfogel also noted that a gift giver has to spend time shopping for a gift, which—unless the giver enjoys spending time shopping—should be added to the cost of gift giving.

As a number of critics of Waldfogel’s analysis have noted, if giving gifts rather than cash makes the recipient worse off and the giver no better off (or worse off if we take into account the cost of the time spent shopping) why has the tradition of giving gifts on holidays and birthdays persisted? Waldfogel argues that for a large fraction of the U.S. population, giving gifts at Christmas is a strong social custom that people are reluctant to break. He believes there is also a social custom against close friends and relatives—parents, siblings, girlfriends, boyfriends, and spouses—giving cash. (Although he believes that it’s socially acceptable for relatives—such as grandparents, aunts, and uncles—who see the gift recipient infrequently to give cash or gift cards).

Some economists have questioned Waldfogel’s results because he surveyed only college students, who are not a representative sample of the population because they are on average younger and come from higher-income families. Because Waldfogel’s students were all enrolled in an economics course, their views may have been affected by what they learned in class. Sarah Solnick, of the University of Vermont, and David Hemenway, of the Harvard University School of Public Health, argue that because most economics students know the economic result that receiving cash as a gift is likely to be preferable to receiving a good, they may have felt social pressure to value their gifts at less than the price paid for them.

To see whether Waldfogel’s including only college students in his survey mattered for his results, Solnick and Hemenway surveyed graduate students and staff at the Harvard School of Public Health as well as people randomly approached at train stations and airports in Boston and Philadelphia. On average the people Solnick and Hemenway surveyed gave their Christmas gifts a value 114 percent higher than the price they estimated the gift giver had paid. In other words, contrary to Waldfogel’s result, gift giving generated a large positive consumer surplus or a welfare gain rather than a welfare loss. The authors speculate that the positive consumer surplus in gift giving may result because the recipient “respects the tastes of the giver, or the item is something the recipient never remembers to get.” Or, perhaps, “The individual wants the item but would feel bad purchasing it for herself. She is grateful to receive it as a gift.”

Solnick and Hemenway’s analysis has also been criticized. Bradley Ruffle and Orit Tykocinski of Ben Gurion University in Israel point out that the order in which questions are asked in a survey can influence the responses. Both Waldfogel and Solnick and Hemenway asked people being surveyed to first estimate what the giver had paid for the gift before asking the value the recipient assigned to the gift. Ruffle and Tykocinski also noted that Solnick and Hemenway changed one question being asked from “the amount of cash such that you are indifferent” between receiving the gift and receiving cash (which is how Waldfogel phrased the question) to the “amount of money that would make you equally happy.” Ruffle and Tykocinski believe this change in wording may also help account for why Solnick and Hemenway’s results differed from Waldfogel’s.

Ruffle and Tykocinski carried out a survey using undergraduate psychology and economics students, varying the wording and the order of the questions. Their results indicated that the wording of the questions mattered, although the order the questions had only a slight effect, and that the psychology students and the economics students did not have significant differences in how much they valued gifts, although psychology students tended to estimate that gift givers had paid a higher price for the gifts. The authors concluded: “Is gift-giving a source of deadweight loss? Our results indicate that it depends critically on how you ask the question and, to a lesser degree, on whom you ask.”

Solnick and Hemenway responded that Ruffle and Tykocinski’s analysis was flawed because, like Waldfogel, they surveyed only undergraduate students: “Ours remains the only study to use adults, living independently, as subjects.” They note that “Ruffle and Tykocinski’s subjects performed poorly in estimating costs,” which may indicate that they lack the experience in buying a large range of goods and so have trouble comparing the value they place on a gift to the cost the giver paid.

John List, of the University of Chicago, and Jason Shogren, of the University of Wyoming, raised the issue of whether the hypothetical nature of the values the gift recipients placed on their gifts mattered. That is, whatever subjective value the recipient gave to a gift, he or she would not actually have the opportunity to sell the gift at a price equal to that value. To test the possibility the recipient’s valuation of a gift would change if the recipient had the opportunity to sell the gift, List and Shogren asked a group of undergraduates to estimate the costs of the gifts they had received for Christmas and to indicate the value they placed on the gifts (just as Waldfogel and the other economists discussed earlier had done). But List and Shogren then added another step by carrying out a so-called random nth price auction of the gifts: “For example, suppose G = 500 gifts overall and #6 was chosen as the random nth price, then only the five lowest-valuation gifts overall would be purchased at the sixth lowest offer.” This somewhat complicated auction design was intended to make it more likely that the students being surveyed would reveal the true value they placed on the gifts.

The results were similar to those found by Solnick and Hemenway in that the recipients put a higher value on the gifts than their estimates of the price the givers paid—so there was positive consumer surplus from gift giving. Their estimate of the welfare gain was significantly smaller than Solnick and Hemenway’s estimate—21 percent to 35 percent versus 114 percent.

Solnick and Hemenway were not entirely convinced by List and Shogren’s findings. They note that because the auction design made it unlikely that the students would actually have to sell their most expensive gifts, the students may have placed a subjective value on these gifts that was too low. In addition, they note that List and Shogren, like Waldfogel, included only college students in their survey.

Joel Waldfogel also replied to List and Shogren, making several of the points he was later to elaborate on in his Scroogenomics book, as discussed above: Basic economic analysis assumes that consumers choose the goods and services they buy to maximize their utility (see Chapter 10 in our textbook), therefore: “If givers, through their choice of gifts, can achieve higher recipient utility than can the recipients themselves, then a fundamental economic assumption is called into question.” List and Shogren compare the value students place on their gifts to the value of receiving cash rather than to the consumer surplus the students would receive from the goods and services they could buy with the cash. Finally, Waldfogel notes that List and Shogren’s results may be affected by what behavioral economists call the endowment effect: The tendency of people to be unwilling to sell a good they already own even if they are offered a price that is greater than the price they would be willing to pay to buy the good if they didn’t already own it. (See the discussion in Chapter 10, Section 10.4 of our textbook.) Because people will require a higher price to sell a good they already own than the price they would pay to buy it, “deadweight loss estimates based on selling prices are much smaller than deadweight loss estimates based on buying prices.”

Even though economists have carried on the debate over the deadweight loss of gift giving in technical terms involving how consumer surplus is best measured, how surveys should be designed, and how best to solicit accurate answers from survey takers, journalists have been intrigued enough by the debate to write it about for general audiences. Josh Barro, who writes for New York magazine and is the son of Harvard economist Robert Barro, wrote a column for the New York Times, “An Economist Goes Christmas Shopping,” in which he observes that the debate among economists over gift giving “makes ordinary people think economists are kind of crazy.” He writes that his father had given him a box of chocolates for Christmas and notes that because he’s on a diet, the gift was “an example of what Mr. Waldfogel warned us about: gift mismatch leading to deadweight loss.” But that he actually ate half the box of chocolates indicated that his father had “identified an item I would not have bought for myself but apparently wanted.” But “now feel I should not have eaten the chocolates, or at least not so many of them in two days.” He concludes that, “The real drag on the economy then isn’t gifts; it’s bad gifts.”

A recent article by Andrew Silver on the Wired website in the United Kingdom notes that a study by academics in India found “an average deadweight loss of about 15 per cent for non-monetary gifts” given during the Hindu festival Diwali. Silver notes that the deadweight loss to gift giving is difficult to avoid because the social custom of gift giving during holidays is very strong in many countries. He concludes, “Sometimes you’ve just got to buckle down and buy something you suspect the recipient won’t value as much as you paid for it.”

Finally, do most economists agree with Waldfogel that there is a significant deadweight loss to gift giving or do they agree with his critics who argue that holiday gift giving actually increases welfare? Although the question has never been asked in a large survey of economists, it was included in a survey of leading economists conducted by the Booth School of Business at the University of Chicago as part of its Initiative on Global Markets (IGM). The IGM regularly surveys a panel of economists on important (although in this case, maybe not so important) economic issues.

A few years ago, they asked their panel whether they agreed with this statement: “Giving specific presents as holiday gifts is inefficient, because recipients could satisfy their preferences much better with cash.” Of the 42 economists who responded to the question, 25 disagreed with the statement, 7 agreed, and 10 were uncertain. Of those who commented, several mentioned a point that Waldfogel had intentionally excluded from his analysis: the sentimental or emotional value that some people attach to giving and receiving presents. For instance, Janet Currie of Princeton noted that: “Gifts serve many functions such as signaling regard and demonstrating social ties with the recipient. Cash transfers don’t do this as well.” Or as Barry Eichengreen of the University of California, Berkeley put it: “Implications of a specific gift (signal it sends, behavioral impact) may give additional utility to either the giver or receiver.” Eric Maskin of Harvard may have stated his reason for disagreeing with the statement most succinctly: “Only an economist could think like this.”

Sources: Joel Waldfogel, “The Deadweight Loss of Christmas,” American Economic Review, Vol. 83, No. 4, December 1993, pp. 328–336; Joel Waldfogel, Scroogenomics: Why You Shouldn’t Buy Presents for the Holidays, Princeton, NJ: Princeton University Press, 2009; Sara J. Solnick and David Hemenway, “The Deadweight Loss of Christmas: Comment,” American Economic Review, Vol. 86, No. 5, December 1996, pp. 1299-1305; Bradley J. Ruffle and Orit Tykocinski, ““The Deadweight Loss of Christmas: Comment,” American Economic Review, Vol. 90, No. 1, March 2000, pp. 319-324; Sara J. Solnick and David Hemenway, “The Deadweight Loss of Christmas: Reply,” American Economic Review, Vol. 90, No. 1, March 2000, pp. 325-326; John A. List and Jason F. Shogren, “The Deadweight Loss of Christmas: Comment,” American Economic Review, Vol. 88, No. 5, December 1998, pp. 1350-1355; Sara J. Solnick and David Hemenway, “The Deadweight Loss of Christmas: Reply,” American Economic Review, Vol. 88, No. 5, December 1998, pp. 1356-1357; Joel Waldfogel, “The Deadweight Loss of Christmas: Reply,” American Economic Review, Vol. 88, No. 5, December 1998, pp. 1358-1360; Tim Hyde, “Did Holiday Gift Giving Just Create a Multi-Billion-Dollar Loss for the Economy?” aeaweb.org, December 28, 2015; Josh Barro, “An Economist Goes Christmas Shopping,” New York Times, December 19, 2014; Andrew Silver, “Economists Want You to Have the Most Boring Christmas Possible,” wired.co.uk, December 17, 2020; and Chicago Booth School of Business, The Initiative on Markets, “Bah, Humbug,” December 17, 2013.

5/15/20 Podcast – Glenn Hubbard & Tony O’Brien Welcome Guest – Texas A&M Economics Professor, Jonathan Meer

Glenn Hubbard and Tony O’Brien continue their podcast series hosting guest – Jonathan Meer, Professor of Economics from Texas A&M University as well as the Director of the Private Enterprise Research Center at Texas A&M. During the conversation, we learn about Jonathan’s teaching over 3,500 students annually in a large online Principles of Microeconomics lecture course. He discusses how his usual online teaching absolutely helped his transition when Texas A&M closed for the semester. He also talks about the state of Higher Education, non-profit giving, as well as some challenges nonprofits face in these uncertain times.

Some notes from this Podcast if you’d like more information:

1. Link to Jonathan Meer’s Youtube video on setting up an online course:

Jonathan Meer of Texas A&M University shares his best practices for teaching economics online.

2. Link to Jonathan’s website page providing links to his research papers on altruism and charitable giving: http://people.tamu.edu/~jmeer/research.html

3. Nontechnical summary of Jonathan’s research with Harvey Rosen on charitable giving: https://www.nber.org/reporter/2018number1/rosen.html

4. Please refer to the Apply the Concept feature from Chapter 2 of Hubbard and O’Brien Economics, 7/E, on the use of market mechanisms to allocate food at the Feeding America charity (for your convenience, we hope to post this shortly so check back).

Please listen & share!