We study a minimal model that has a driven-dissipative quantum phase transition, namely a Kerr non-linear oscillator subject to driving and dissipation. Using mean-field theory, exact diagonalization, and the Keldysh formalism, we analyze the critical phenomena in this system, showing which aspects can be captured by each approach and how the approaches complement each other. Then critical scaling and finite-size scaling are calculated analytically using the quantum Langevin equation. The physics contained in this simple model is surprisingly rich: it includes a continuous phase transition, Z2 symmetry breaking, PT symmetry, state squeezing, and critical fluctuations. Due to its simplicity and solvability, this model can serve as a paradigm for exploration of open quantum many-body physics.