Much of medical risk prediction involves externally derived prediction equations, nomograms, and point-based risk scores. These settings are vulnerable to misleading findings of incremental value based on versions of the net reclassification index (NRI) in common use. By applying non-nested models and point-based risk scores in the setting of stroke risk prediction in patients with atrial fibrillation (AF), we demonstrate current recommendations for presentation and interpretation of the NRI. We emphasize pitfalls that are likely to occur with point-based risk scores that are easy to neglect when statistical methodology is focused on continuous models. In order to make appropriate decisions about risk prediction and personalized medicine, physicians, researchers, and policy makers need to understand the strengths and limitations of the NRI.