We present a general framework for inverse design of nanopatterned surfaces that maximize spatially averaged surface-enhanced Raman (SERS) spectra from molecules distributed randomly throughout a material or fluid, building upon a recently proposed trace formulation for optimizing incoherent emission. This leads to radically different designs than optimizing SERS emission at a single known location, as we illustrate using several 2D design problems addressing effects of hot-spot density, angular selectivity, and nonlinear damage. We obtain optimized structures that perform about 4 × better than coating with optimized spheres or bowtie structures and about 20 × better when the nonlinear damage effects are included.