A sol-gel process together with a low-temperature sintering technique was applied for preparing Mn2+ activated Li2MgSiO4 green phosphor. Phase structure information was derived from the Rietveld refinement on its powder X-ray diffraction pattern. The luminescence performance was evaluated by characterizing the photoluminescence (PL) spectral evolution with Mn2+ concentration and in the temperature range of 300–500 K. The optimal Mn2+ doping level was 4 at% and the main PL quenching mechanism was identified to be the electric dipolar-dipolar interaction. The temperature-dependent PL spectra showed broad asymmetric line-profiles, and the underlying phonon effect was revealed from both experimental and theoretical works. The predominant phonon energy was determined to be 25 meV, and the luminescence quenching at high temperatures was mainly attributed to the six-phonon coupling interaction with the Mn2+ optical center, which was further confirmed from the simulation on PL spectral lineshape. These obtained results may provide us an unprecedented approach to study the phonon effect on the luminescence characteristic for the phosphors.