Accepted and Published Papers:
Evaluating Behavioral Incentive Compatibility: Insights from Experiments
(with David Danz and Lise Vesterlund.
Journal of Economic Perspectives, Fall, 2024, 38(4): p.131-54
[+] Abstract Incentive compatibility is core to mechanism design. The success of auctions, matching algorithms, and voting systems all hinge on the ability to select incentives that make it in the individual's interest to reveal their type. But how do we test whether a mechanism that is designed to be incentive compatible is actually so in practice, particularly when faced with boundedly rational agents with nonstandard preferences? We review the many experimental tests that have been designed to assess behavioral incentive compatibility, separating them into two categories: indirect tests that evaluate behavior within the mechanism, and direct tests that assess how participants respond to the mechanism's incentives. Using belief elicitation as a running example, we show that the most popular elicitations are not behaviorally incentive compatible. In fact, the incentives used under these elicitations discourage rather than encourage truthful revelation.
Additional details:
Limits on Regret as a Tool for Incentive Design
(with Alex Imas and Felipe Araujo).
Accepted at Journal of Political Economy Microeconomics, July, 2024
[+] Abstract
We demonstrate the pitfalls when extrapolating behavioral findings across different contexts and decision environments. We focus on regret theory and the use of ``regret lotteries'' for motivating behavior change. Here, findings from one-shot settings have been used to promote regret as a tool to boost incentives in recurrent decisions across many settings. Using theory and experiments, we replicate regret lotteries as the superior one-shot incentive; however, for repeated decisions the comparative static is entirely reversed. Moreover, the effects are extremely sensitive to details of regret implementation. Our results suggest caution should be used when designing incentive schemes that exploit regret.
Additional details:
Testing Models of Strategic Uncertainty: Equilibrium Selection in Repeated Games
(with Marta Boczon, Emanuel Vespa and Taylor Weidman)
Accepted at Journal of the European Economic Association, June 2024
[+] Abstract
In repeated-game applications, where both the collusive and noncollusive outcomes can be supported as equilibria, it is crucial for researchers to understand when each equilibria
is more likely to result. Controlled experiments have provided a selection criterion for the two-player repeated prisoner's dilemma. The theoretical selection criterion uses game
primitives to measure the set of beliefs such that an agent prefers to defect in all rounds relative to cooperating conditionally. The set of such beliefs is large when uncertainty
with respect to what others will do--strategic uncertainty--is large, and the prediction is that a noncooperative outcome would emerge. In this paper, we experimentally test this
model of selection and its underlying mechanism of strategic uncertainty by manipulating the total number of players. Our results affirm the model as a tool for predicting when tacit
collusion is likely/unlikely to be successful. Extending the analysis, we corroborate the findings and mechanism of the model, in alternative settings, and for the decisions of non-human
AI agents.
Goals, Constraints, and Transparently Fair Assignment: A Field Study of the UEFA Champions League
(with Marta Boczon)
Management Science, June 2023, 69(6): p.3474-01
[+] Abstract We analyze the design of a randomization procedure in a field setting with high stakes and substantial public interest: matching sports teams in the UEFA Champions League. While striving for fairness in the chosen lottery - giving teams similar distributions over potential partners - the designers seek to balance two conflicting forces: (i) imposing a series of combinatorially complex constraints on the feasible matches; (ii) designing an easy-to-understand and credible randomization. We document the tournament's solution, which focuses on sequences of uniform draws over each element in the final match, assisted by a computer to form the support for each draw. We first show that the constraints' effects within this procedure are substantial, with shifts in expected prizes of up to a million euro and large distortions in match likelihoods of otherwise comparable team pairs. However, examining all possible counterfactual lotteries over the feasible assignments, we show that the generated inequalities are for the most part unavoidable, that the tournament design is close to a constrained-best. In two extensions we outline how substantially fairer randomizations are possible when the constraints are weakened, and how the developed procedure can be adopted to more-general settings.
Additional details:
Belief Elicitation and Behavioral Incentive Compatibility
(with David Danz and Lise Vesterlund)
American Economic Review, September 2022, 112(9): p.2851-83 (lead article)
[+] Abstract
Subjective beliefs are crucial for economic inference, yet behavior can challenge the elicitation. We propose that belief elicitation should be incentive compatible not only theoretically but also in a de facto behavioral sense. To demonstrate, we show that the binarized scoring rule, a state-of-the-art elicitation, violates two weak conditions for behavioral incentive compatibility: (i) within the elicitation, information on the incentives increases deviations from truthful reporting; and (ii) in a pure choice over the set of incentives, most deviate from the theorized maximizer. Moreover, we document that deviations are systematic and center-biased, and that the elicited beliefs substantially distort inference.
Top of the Batch: Interviews and the Match
(with Federico Echenique, Ruy Gonzalez and Leeat Yariv)
American Economic Review: Insights June 2022 4(2): p.223-38 .
[+] Abstract
Most doctors in the NRMP match with one of their most-preferred internship programs. However, surveys indicate doctors' preferences are similar, suggesting a puzzle: how can so many doctors match with their top choices when positions are scarce? We provide one possible explanation. We show that the patterns in the NRMP data may be an artifact of the interview process that precedes the match. Our study highlights the importance of understanding market interactions occurring before and after a matching clearinghouse. It casts doubts on analyses of clearinghouses that take reported preferences at face value.
The Times They are a-Changing: Experimenting with Dynamic Adverse Selection
(with Felipe Araujo and Stephanie Wang).
American Economic Journal: Microeconomics, November 2021, 13(4): p.1-22
[+] Abstract
We examine a common-value dynamic matching environment where adverse selection accrues slowly over time. Theoretical best responses are therefore time varying, and where the prior experimental literature suggests that sequential environments might lead to greater understanding of adverse selection in this dynamic setting. However, while a sophisticated minority in our experiment do condition on time and are close to a best response, the majority use a stationary response, even after extended experience. In an environment with persistent uncertainty, our results indicate that sequentiality is insufficient for the large majority of participants to recognize the effects of adverse selection.
Information Transmission under the Shadow of the Future: An Experiment
(with Emanuel Vespa).
American Economic Journal: Microeconomics, November 2020, 12(4): p.75-98
[+] AbstractWe experimentally examine how information transmission functions in an ongoing relationship. Where the one-shot cheap-talk literature documents substantial over-communication and preferences for honesty, in our repeated setting the outcomes are more consistent with uninformative babbling outcomes. This is particularly surprising as honest revelation is supportable as an equilibrium outcome in our repeated setting. We show that inefficient outcomes are driven by a coordination failure on how to distribute the gains from information sharing. However, when agents can coordinate on the payment of an "information rent," honest revelation emerges.
Additional details:
Experimenting with Incentives for Information Transmission: Quantity Versus Quality
(with Jonathan Lafky)
Journal of Economic Behavior and Organization, January 2020, 169: p.314-31 (Working Paper)
[+] Abstract People share their experiences of goods and services online, through reviews, ratings and endorsements on social networks, potentially generating welfare-improving information that can help subsequent consumers make better, more informed decisions. While the economics literature has focused on questions of alignment and the intensive quality of provided information, another tension is extensive: in the absence of an incentive, many might choose not to provide information at all. We study three different incentives that encourage information transmission on the extensive margin, examining the tradeoffs between the quality and quantity of information. Our findings indicate substantial efficiency gains can be made relative to no incentives, even when the incentives damage the preference alignment between those sending and receiving information. In particular, our results point to a partially aligned incentive (similar to a referral or sales commission) as robustly encouraging the provision of information while not producing substantial reductions in quality.
Experimenter Demand Effects
(with Jon De Quidt) and Lise Vesterlund
Chapter in Handbook of research methods & applications in experimental economics (eds. A Schram & A Ule), Edward Elgar
[+] Abstract A study's internal and external validity is threatened by experimenter demand effects. This threat is taken seriously by experimental economists, who have developed a number of best practices to suppress or eliminate the potential role of such effects. We outline these best practices and review the literature to show that they are followed in the vast majority of published work. This adherence to best practice likely contributes to the limited evidence of experimenter demand effects uncovered in the literature. Specifically, we are not aware of examples where demand effects have been shown to influence the qualitative inference from a study. While good design goes a long way towards reducing the potential for experimenter demand effects, a complementary option, presented in our final section, is to derive bounds on the effect.
Experimenting with the Transition Rule in Dynamic Games
(with Emanuel Vespa)
Quantitative Economics, November 2019, 10(4): p.1825-49 (open access)
[+] Abstract
In dynamic environments where the strategic setting evolves across time, the specific rule governing the transitions can substantially alter the incentives agents face. This is particularly true when history-dependent strategies are used. In a laboratory study, we examine whether subjects respond to the transition rule and internalize its effects on continuation values. Our main comparison is between an endogenous transition where future states directly depend on current choices, and exogenous transitions where the future environment is random and independent of actions. Our evidence shows that subjects readily internalize the effect of the dynamic game transition rule on their incentives, in line with history-dependent theoretical predictions.
Additional details:
Precursor Working Paper: Experimenting with Equilibrium Selection in Dynamic Games
A Proposal to Organize and Promote Replications
(with Lucas Coffman, Muriel Niederle)
American Economic Review (Papers and Proceedings), May 2017, 107(5), pp.41-45
[+] Abstract
We make a two-pronged proposal to (i) strengthen the incentivizes for replication work and (ii) better organize and draw attention to the replications that are conducted. First we propose that top journals publish short "replication reports." These reports could summarize novel work replicating an existing high-impact paper, or they could highlight a replication result embedded in a wider-scope published paper. Second, we suggest incentivizing replications with the currency of our profession: citations. Enforcing a norm of citing replication work alongside the original would provide incentives for replications to both authors and journals.
Assessing the Rate of Replication in Economics
(with James Berry,Lucas Coffman, Rania Gihleb, Douglas Hanley)
American Economic Review (Papers and Proceedings), May 2017, 107(5), pp. 27-31
[+] Abstract
We assess the rate of replication for empirical papers in the 2010 American Economic Review. Across seventy empirical papers, we find that 29 percent have one or more citation that partially replicates the original result. While only a minority of papers has a published replication, a majority (sixty percent) have either a replication, robustness test or an extension. Surveying authors within the literature we find substantial uncertainty over the number of extant replications.
At-will Relationships: How an Option to Walk Away Affects Cooperation and Efficiency
(with Hong Wu).
Games & Economic Behavior, March 2017, 102: p.487-507 .
[+] Abstract
We theoretically and experimentally examine the effects from adding a simple, empirically relevant action to a repeated partnership, the option to walk away. Manipulating both the value of the outside option, and its relative distribution among the partners, we examine the behavior of human subjects in a repeated prisoners' dilemma. In particular, we examine the degree of cooperation and the form of punishments used. Our findings indicate that cooperation rates are broadly unaffected by the value of the common outside option, but that the selection of supporting punishments - in-relationship defections or walking-away - are dictated by individual rationality. In contrast to the symmetric results, when outside options for partners are asymmetric we find stark selection effects over cooperation, with the potential for very high and very low efficiency, dependent on the precise division rule.
Additional details:
Clearinghouses for Two-Sided Matching: An Experimental Study
(with Federico Echenique, Leeat Yariv)
Quantitative Economics, July 2016, 7(2): p.449-82 (open access)
[+] Abstract
We experimentally study the Gale and Shapley (1962) mechanism, utilized in a wide set of applications, most prominently the National Resident Matching Program (NRMP). Several insights come out of our analysis: First, only 48% of our observed outcomes are stable. Among those, a large majority culminates at the receivers-optimal stable matching. Second, receivers rarely truncate their true preferences; it is the proposers who do not make offers in order of their preference, frequently skipping potential partners. Third, market characteristics affect behavior: both the cardinal representation and core size influence whether laboratory outcomes arestable. We conclude by using our controlled results to shed light on a number of stylized facts we derive from new NRMP survey and outcome data, and to explain the small cores previously documented for the NRMP.
The Slider Task: An Example of Restricted Inference on Incentive Effects
(with Felipe Araujo, Erin Carbone, Lynn Connell-Price, Marli Dunietz, Ania Jaroszewicz, Rachel Landsman, Diego Lamé, Lise Vesterlund and Stephanie Wang)
Journal of Economic Science Association, 2016, 2(1), p.1-12 (lead article, Working Paper)
[+] Abstract
Real-effort experiments are frequently used when examining a response to incentives. For a real-effort task to be well suited for such an exercise its measureable output must be sufficiently elastic over the incentives considered. The popular slider task in Gill and Prowse (2012) has been characterized as satisfying this requirement, and the task is increasingly used to investigate the response to incentives. However, a between-subject examination of the slider task's response to incentives has not been conducted. We provide such an examination with three different piece-rate incentives: half a cent, two cents, and eight cents per slider completed. We find only a small increase in performance: despite a 1,500 percent increase in the incentives, output only increases by 5 percent. With such an inelastic response we caution that for typical experimental sample sizes and incentives the slider task is unlikely to demonstrate a meaningful and statistically significant performance response.
Additional details:
Expanded working paperThe Effect of Incentives on Real Effort: Evidence from the Slider Task Representative Instructions
Communication With Multiple Senders: An Experiment
Quantitative Economics, March 2016, 7(1): p.1-36 (open access), (lead article)
(with Emanuel Vespa).
[+] Abstract
We implement multi-sender cheap talk in the laboratory. While full-information transmission is not theoretically feasible in the standard one-sender one-dimension model, in this setting with more senders and dimensions, full revelation is generically a robust equilibrium outcome. Our experimental results indicate that fully revealing outcomes are selected in particular settings, but that partial-information transmission is the norm. We uncover a number of behavioral patterns: Senders follow exaggeration strategies, qualitatively similar to those predicted by a special case for the fully revealing equilibrium. Receivers on the other hand follow differing heuristics depending on the senders' biases, which are not always sequentially rational. When full revelation is observed it is largely due to the intersection of the receiver heuristics with the equilibrium response.
Additional details:
Supplementary Material to QE version
Working Paper (September 2011) , additional results on treatments with non-full-rank bias matrices will be reported in an (in preparation) companion paper: Competition and Communication: An Experiment.. Link to journal page is here.A Field Study on Matching with Network externalities
(with Mariagiovanna Baccara, Ayse Imrohoroglu, Leeat Yariv.)
American Economic Review, August 2012, 102(5): p.1773-1804 (lead article)
[+] Abstract
We study the effects of network externalities on a unique matching protocol for faculty in a large U.S. professional school to offices in a new building. We collected institutional, web, and survey data on faculty's attributes and choices. We first identify the different layers of the social network: institutional affiliation, coauthorships, and friendships. We demonstrate and quantify the effects of network externalities on choices and outcomes. Furthermore, we disentangle the different layers of the social network and quantify their relative impact. Finally, we assess the matching protocol from a welfare perspective. Our study suggests the importance and feasibility of accounting for network externalities in general assignment problems and evaluates a set of techniques that can be employed to this end.
Additional details:
A game theoretic approach to multimodal communication
(with Mark Dean, and James Higham.)
Behavioral Ecology & Sociobiology, 2013, 69(9), p.1399-1415
[+] Abstract
Over the last few decades the animal communication community has become increasingly aware that much communication occurs using multiple signals in multiple modalities. The majority of this work has been empirical, with less theoretical work on the advantages conferred by such communication. In the present paper we ask: Why should animals communicate with multiple signals in multiple modalities? To tackle this question we use game theoretic techniques, and highlight developments in the economic signaling literature that might offer insight into biological problems. We start by establishing a signaling game, and investigate signal honesty under two prevailing paradigms of honest communication--costly signaling and cheap talk. In both paradigms, without further constraint, it is simple to show that anything that can be achieved with multiple signals can be achieved with one. We go on to investigate different sets of possible constraints that may make multiple signals and multimodal signals in particular more likely to evolve. We suggest that constraints on cost functions and bandwidths, orthogonal noise across modalities, strategically distinct modes, multiple qualities, multiple signalers, and multiple audiences, all provide biologically plausible scenarios that theoretically favor multiple and multimodal signaling.