We address the removal of canvas artifacts from high-resolution digital photographs and X-ray images of paintings on canvas. Both imaging modalities are common investigative tools in art history and art conservation. Canvas artifacts manifest themselves very differently according to the acquisition modality; they can hamper the visual reading of the painting by art experts, for instance, in preparing a restoration campaign. Computer-aided canvas removal is desirable for restorers when the painting on canvas they are preparing to restore has acquired over the years a much more salient texture. We propose a new algorithm that combines a cartoon-texture decomposition method with adaptive multiscale thresholding in the frequency domain to isolate and suppress the canvas components. To illustrate the strength of the proposed method, we provide various examples, for acquisitions in both imaging modalities, for paintings with different types of canvas and from different periods. The proposed algorithm outperforms previous methods proposed for visual photographs such as morphological component analysis and Wiener filtering and it also works for the digital removal of canvas artifacts in X-ray images.