The quantum erasure channel is the simplest example of a quantum communication channel and its information capacity is known precisely. The subclass of quantum error-correcting codes called stabilizer codes is known to contain capacity-achieving sequences for the quantum erasure channel, but no efficient method is known to construct these sequences. In this article, we explicitly describe a capacity-achieving code sequence for the quantum erasure channel. In particular, we show that Calderbank-Shor-Steane (CSS) stabilizer codes constructed from self-orthogonal binary linear codes are capacity-achieving on the quantum erasure channel if the binary linear codes are capacity-achieving on the binary erasure channel. Recently, Reed-Muller codes were shown to achieve capacity on classical erasure channels. Using this, we show that CSS codes constructed from binary Reed-Muller codes achieve the capacity of the quantum erasure channel. The capacity-achieving nature of these CSS codes is also explained from a GF(4) perspective.