Skip to main content

Experimentation and Start-up Performance: Evidence from A/B Testing

Publication ,  Journal Article
Koning, R; Hasan, S; Chatterji, A
Published in: Management Science
September 1, 2022

Recent scholarship argues that experimentation should be the organizing principle for entrepreneurial strategy. Experimentation leads to organizational learning, which drives improvements in firm performance. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of testing business ideas. Our results provide the first evidence on how digital experimentation affects a large sample of high-technology start-ups using data that tracks their growth, technology use, and products. We find that, although relatively few firms adopt A/B testing, among those that do, performance improves by 30%–100% after a year of use. We then argue that this substantial effect and relatively low adoption rate arises because start-ups do not only test one-off incremental changes, but also use A/B testing as part of a broader strategy of experimentation. Qualitative insights and additional quantitative analyses show that experimentation improves organizational learning, which helps start-ups develop more new products, identify and scale promising ideas, and fail faster when they receive negative signals. These findings inform the literatures on entrepreneurial strategy, organizational learning, and data-driven decision making.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Management Science

DOI

EISSN

1526-5501

ISSN

0025-1909

Publication Date

September 1, 2022

Volume

68

Issue

9

Start / End Page

6434 / 6453

Related Subject Headings

  • Operations Research
  • 46 Information and computing sciences
  • 38 Economics
  • 35 Commerce, management, tourism and services
  • 15 Commerce, Management, Tourism and Services
  • 08 Information and Computing Sciences
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Koning, R., Hasan, S., & Chatterji, A. (2022). Experimentation and Start-up Performance: Evidence from A/B Testing. Management Science, 68(9), 6434–6453. https://doi.org/10.1287/mnsc.2021.4209
Koning, R., S. Hasan, and A. Chatterji. “Experimentation and Start-up Performance: Evidence from A/B Testing.” Management Science 68, no. 9 (September 1, 2022): 6434–53. https://doi.org/10.1287/mnsc.2021.4209.
Koning R, Hasan S, Chatterji A. Experimentation and Start-up Performance: Evidence from A/B Testing. Management Science. 2022 Sep 1;68(9):6434–53.
Koning, R., et al. “Experimentation and Start-up Performance: Evidence from A/B Testing.” Management Science, vol. 68, no. 9, Sept. 2022, pp. 6434–53. Scopus, doi:10.1287/mnsc.2021.4209.
Koning R, Hasan S, Chatterji A. Experimentation and Start-up Performance: Evidence from A/B Testing. Management Science. 2022 Sep 1;68(9):6434–6453.

Published In

Management Science

DOI

EISSN

1526-5501

ISSN

0025-1909

Publication Date

September 1, 2022

Volume

68

Issue

9

Start / End Page

6434 / 6453

Related Subject Headings

  • Operations Research
  • 46 Information and computing sciences
  • 38 Economics
  • 35 Commerce, management, tourism and services
  • 15 Commerce, Management, Tourism and Services
  • 08 Information and Computing Sciences