Centromeres are built on repetitive DNA sequences (CenDNA) and a specific chromatin enriched with the histone H3 variant CENP-A, the epigenetic mark that identifies centromere position. Here, we interrogate the importance of CenDNA in centromere specification by developing a system to rapidly remove and reactivate CENP-A (CENP-AOFF/ON ). Using this system, we define the temporal cascade of events necessary to maintain centromere position. We unveil that CENP-B bound to CenDNA provides memory for maintenance on human centromeres by promoting de novo CENP-A deposition. Indeed, lack of CENP-B favors neocentromere formation under selective pressure. Occasionally, CENP-B triggers centromere re-activation initiated by CENP-C, but not CENP-A, recruitment at both ectopic and native centromeres. This is then sufficient to initiate the CENP-A-based epigenetic loop. Finally, we identify a population of CENP-A-negative, CENP-B/C-positive resting CD4+ T cells capable to re-express and reassembles CENP-A upon cell cycle entry, demonstrating the physiological importance of the genetic memory.