INTRODUCTION: Family history (FH) of Alzheimer's disease (AD) affects mitochondrial function and may modulate effects of translocase of the outer mitochondrial membrane 40 kDa (TOMM40) rs10524523 ('523) poly-T length on memory decline. METHODS: For 912 nonapolipoprotein ε4 middle-aged adults and 365 aged adults across the AD spectrum, linear mixed models gauged FH and TOMM40 '523 interactions on memory and global cognition between baseline and up to 10 years later. A cerebrospinal fluid mitochondrial function biomarker was also assessed. RESULTS: For FH negative participants, gene-dose preservation of memory and global cognition was seen for "very long" versus "short" carriers. For FH positive, an opposite gene-dose decline was seen for very long versus short carriers. Maternal FH was a stronger predictor in aged, but not middle-aged, participants. Similar gene-dose effects were seen for the mitochondrial biomarker aspartate aminotransferase. DISCUSSION: These results may clarify conflicting findings on TOMM40 poly-T length and AD-related decline.