A joint petrophysical and structural inversion method for electromagnetic (EM) and seismic data based on the volume integral equation (VIE) is proposed in this paper. In the forward EM problem, only the contrast of conductivity is solved by the electric field integral equation method. However, in the forward seismic problem, both the contrasts of velocity and mass density are solved by the combined field VIE method. Both forward solvers are accelerated by the fast Fourier transform. In the inversion problem, by using the petrophysical equations about the porosity and saturation and applying the chain rule, we fuse the EM and seismic data and construct the joint petrophysical inversion equations, which can be solved by the variational Born iteration method. Then, in order to further enhance the reconstructed results of the joint petrophysical inversion, we enforce the structural similarity constraint between porosity and water saturation and add the cross-gradient function to the joint petrophysical inversion cost function. Two typical geophysical models based on the remote sensing measurement are used to validate the proposed methods. One is the cross-well model, and the other is the marine surface exploration model. The advantage of the joint inversion compared with the separate inversion is evaluated based on the resolution and the data misfits of the reconstructed profiles as well as the antinoise ability.