We argue that a critical component of designing comprehensible intelligent systems is finding the right applications for intelligence and designing intelligent solutions toward those applications. While we do not refute the value of good attention to later stages of human-centered design such as the application of human interface design principles and usability testing as methods for improving comprehensibility, there must also be significant attention to understanding problems in the context of use and how intelligence systems can best address those problems. In light of supporting naturalistic decision-making, we present a review of task analytic and qualitative research techniques that may be useful for better understanding problems in context that will support the design of more comprehensible intelligent systems. © 2009 IEEE.