Photorefractive volume holography for processing ultrashort optical pulses carrying spatial, temporal and spatio-temporal optical information is introduced. These new holographic methods can process 4-dimensional information that in addition to the 3 spatial coordinates also include the temporal evolution of optical signals on nanosecond to femtosecond scale. Photorefractive volume holographic materials provide the medium necessary for recording and reconstruction in real-time. Applications of direct time domain and spectral domain holography for image processing, temporal matched filtering, optical pulse shaping, 3-D optical storage, and optical interconnects are discussed. Furthermore, the combined space-time holographic processing that allows the conversion between spatial and temporal optical information carrying channels is introduced. This method is used to demonstrate experimentally parallel-to-serial and serial-to-parallel data conversion for 1-D images and image-format data transmission. This holographic processors provide the advantages of self-referenced signal transmission and self-compensation for optical dispersion induced by the holographic materials, communication channel, as well as other optical components. Finally, future research directions for optical information processing with complex spatio-temporal signals are identified and discussed