© Copyright 2018 SPIE. Differentiating material anomalies requires a measurement system that can reliably inform the user/classifier of pertinent material characteristics. In past work, we have developed a simulation framework capable of making simulated x-ray transmission and scatter measurements of virtual baggage. Using this simulated data, we have demonstrated how an information-theoretic approach to x-ray system design and analysis provides insight into system performance. Moreover, we have shown how performance limits relate to architectural variations in source fluence, view number, spectral resolution, spatial resolution, etc. However, our previous investigations did not include material variability in the description of the materials which make up the virtual baggage. One would expect the material variability to dramatically affect the results of the information-theoretic metric, and thus we now include it in our analysis. Previously, material information was captured as energy-dependent mean attenuation values. Because of this, material differentiation can always become easier with an improvement in SNR. When there is no variation to obscure class differences, improvements in SNR will indefinitely improve performance. Therefore, we saw a monotonic increase of the metric with source fluence. However there is inherent variability in materials from chemical impurities, texturing, or macroscopic variation. When this variability is accounted for, we better understand system performance limits at higher SNR as well as better represent the distributions of material characteristics. We will report on the analysis of real world system geometries and the fundamental limits of performance limits after incorporating these material variability improvements.