The impact of bacterial species on outcome in bloodstream infections (BSI) is incompletely understood. We evaluated the impact of bacterial species on BSI mortality, with adjustment for patient, bacterial, and treatment factors. From 2002 to 2015, all adult inpatients with monomicrobial BSI caused by Staphylococcus aureus or Gram-negative bacteria at Duke University Medical Center were prospectively enrolled. Kaplan-Meier curves and multivariable Cox regression with propensity score models were used to examine species-specific bacterial BSI mortality. Of the 2,659 enrolled patients, 999 (38%) were infected with S. aureus, and 1,660 (62%) were infected with Gram-negative bacteria. Among patients with Gram-negative BSI, Enterobacteriaceae (81% [1,343/1,660]) were most commonly isolated, followed by non-lactose-fermenting Gram-negative bacteria (16% [262/1,660]). Of the 999 S. aureus BSI isolates, 507 (51%) were methicillin resistant. Of the 1,660 Gram-negative BSI isolates, 500 (30%) were multidrug resistant. The unadjusted time-to-mortality among patients with Gram-negative BSI was shorter than that of patients with S. aureus BSI (P = 0.003), due to increased mortality in patients with non-lactose-fermenting Gram-negative BSI generally (P < 0.0001) and Pseudomonas aeruginosa BSI (n = 158) in particular (P < 0.0001). After adjustment for patient demographics, medical comorbidities, bacterial antibiotic resistance, timing of appropriate antibiotic therapy, and source control in patients with line-associated BSI, P. aeruginosa BSI remained significantly associated with increased mortality (hazard ratio = 1.435; 95% confidence interval = 1.043 to 1.933; P = 0.02). P. aeruginosa BSI was associated with increased mortality relative to S. aureus or other Gram-negative BSI. This effect persisted after adjustment for patient, bacterial, and treatment factors.