Working Papers:

The Experimenters' Dilemma: Inferential Preferences over Populations

(with Neeraja Gupta and Luca Rigotti)

July 2021

We compare three populations commonly used in experiments by economists and other social scientists: undergraduate students at a physical location (lab), Amazon's Mechanical Turk (MTurk), and Prolific. The comparison is made along three dimensions: the noise in the data due to inattention, the cost per observation, and the elasticity of response. We draw samples from each population, examining decisions in four one-shot games with varying tensions between the individual and socially efficient choices. When there is no tension, where individual and pro-social incentives coincide, noisy behavior accounts for 60% of the observations on MTurk, 19% on Prolific, and 14% for the lab. Taking costs into account, if noisy data is the only concern Prolific dominates from an inferential power point of view, combining relatively low noise with a cost per observation one fifth of the lab's. However, because the lab population is more sensitive to treatment, across our main PD game comparison the lab still outperforms both Prolific and MTurk.

Testing Models of Strategic Uncertainty: Equilibrium Selection in Repeated Games

(with Emanuel Vespa and Taylor Weidman)

January 2021, revise & resubmit at Journal of the European Economic Association

In repeated-game applications where both the collusive and non-collusive outcomes can be supported as equilibria, researchers must resolve underlying selection questions if theory will be used to understand counterfactual policies. One guide to selection, based on clear theoretical underpinnings, has shown promise in predicting when collusive outcomes will emerge in controlled repeated-game experiments. In this paper we both expand upon and experimentally test this model of selection, and its underlying mechanism: strategic uncertainty. Adding an additional source of strategic uncertainty (the number of players) to the more-standard payoff sources, we stress test the model. Our results affirm the model as a tool for predicting when tacit collusion is likely/unlikely to be successful. Extending the analysis, we corroborate the mechanism of the model. When we remove strategic uncertainty through an explicit coordination device, the model no longer predicts the selected equilibrium.

Additional details:

Online Appendices

Belief Elicitation: Limiting Truth Telling with Information on Incentives

(with David Danz and Lise Vesterlund)

August 2020, revise & resubmit at American Economic Review

We study truth telling within the current state-of-the-art mechanism for belief elicitation and examine how information on incentives affects reports on a known objective prior. We find that transparent information on incentives gives rise to error rates in excess of 40 percent, and that only 15 percent of participants consistently report the truth. False reports are conservative and appear to result from a biased perception of the BSR incentives. While attempts to debias are somewhat successful, the highest degree of truth telling occurs when information on quantitative incentives is withheld. Perversely the mechanismís incentives are shown to decrease truthful reporting.

Additional details:


Preference Reversals between One-Shot and Repeated Decisions: The Case of Regret

(with Alex Imas and Diego Lamé).

August, 2020, revise & resubmit at Management Science

We demonstrate potential pitfalls when extrapolating behavioral findings identified from one-shot choices to repeated settings. As a case study, we examine the use of "regret lotteries" as a behavioral tool to boost motivation. Based on findings from one-shot settings, presenting counterfactual information generates the potential for regret, which can be used to increase a lottery's value. This result has motivated the increasing use of regret lotteries in the field to incentivize recurrent decisions like exercise and compliance with company directives. Using a controlled experiment we show that while regret lotteries are the superior motivational tool in one-shot decisions, for repeated decisions the effect is entirely reversed. These findings have implications for incentive and policy design, highlighting the scope for error when extrapolating one-shot findings to inherently repeated settings.

Additional details:


Goals, Constraints, and Public Assignment: A Field Study of the UEFA Champions League

(with Marta Boczon)

Febuary 2020, revise & resubmit at Management Science

We analyze a matching mechanism developed to solve a complex constrained-assignment problem, where the entire randomization needs to be transparent to an outside observer. The application (the UEFA Champions League tournament draw) is high stakes; participants' expected outcomes can shift by millions of euro depending on the draw's realization. Moreover, for a centralized assignment with nontrivial constraints, the draw brings in a huge audience, with at least a million following along from around the world. After quantifying the constraint effects, we turn to a normative assessment: Are there better, fairer procedures? Relying upon a combination of theory, structural estimation, and simulation, we outline a quantitative methodology aimed at assessing the transparent assignment procedure developed by UEFA. Our analysis indicates that the mechanism comes quantitatively close to a constrained-best in fairness terms. Moreover, we demonstrate that while substantially better mechanisms do not exist given the current constraint structure, it is possible to substantially reduce matching distortions by only marginally slacking the constraints.

Additional details:

Online Appendix

Costly Communication in Groups: Theory and an Experiment

February 2016

I develop a novel model of group-based communication in which group members communicate with one another. Communication is costly in the sense that group members who choose to send or listen to messages incur costs. Equilibrium strategies have an intuitive characterization - those with the best information send, those with the worst information receive. Free-riding leads to less information exchange than is optimal, but a simple system of transfers and subsidies can correct this. Examining the model's predictions with an experiment I find that subjects over-communicate when costs are high, but fail to benefit from this as much as they should. Additionally, I find that listening costs are more harmful to welfare, in contrast with the theory which indicates sending costs.

Paired-Uniform Scoring: Implementing a binarized scoring rule with non-mathematical language

(with Emanuel Vespa).

October, 2017

We outline a mechanism for eliciting probabilities using two uniform random numbers that is equivalent to the binarized scoring rule (BSR). Though our implementation is simple to describe and has a non-mathematical explanation, it retains the desirable theoretical features of the BSR. Moreover, we show that a discretized version with evenly-spaced reporting intervals can be implemented in the field with no more equipment than a pair of dice.

Work in Progress:

Laws of Large Numbers and Risk Preferences

(with Felipe Araujo and Alex Imas)

Testing Qualitative Effects with Experimenter Demand

(with David Danz, Lise Vesterlund and graduate experimental class)

Honesty on the Margins

(with Simon Halliday and Jonathan Lafky)

Strategic Uncertainty in Dynamic Games

(with Emanuel Vespa)

Competition and Communication: An Experiment

(with Emanuel Vespa)

Replicating Moral wiggling across domains

(with David Danz, Lise Vesterlund and graduate experimental class)

An Elicitation Horse Race (Where the Blinkered Horse Win by a Nose)

(with David Danz, Lise Vesterlund and and graduate experimental class)

Paying it forward: An experiment on an experimental college financing

(with David Danz, David Huffman, Lise Vesterlund and Stephanie Wang)