Organizational decision-making and the returns to experimentation
Many organizations have embraced formal experimentation, i.e., A/B testing, to improve the performance of their products and services. Experimentation, some have argued, should democratize innovation inside organizations by creating a platform to test new ideas, regardless of origin. In this article, we argue that experimentation’s promise hinges on having the proper organizational decision-making process that encourages innovation while mitigating the risk of unanticipated failures. We study this question by developing a model of experimentation inside organizations, where decisions to implement are either centralized or decentralized—a tension identified by practitioners and scholars alike. Organizations with centralized mechanisms that rely too much on the input of other teams benefit least from experimentation, as do ones with completely decentralized ones. In contrast, organizations with mostly decentralized decisions, with a single authority that sets consistent thresholds for implementation, achieve growth but with less downside risk. Thus, without considering the organizational decision-making structure, the benefits of experimentation may be limited.
