Sirtuins are NAD+-dependent protein deacylases that regulate several aspects of metabolism and aging. In contrast to the other mammalian sirtuins, the primary enzymatic activity of mitochondrial sirtuin 4 (SIRT4) and its overall role in metabolic control have remained enigmatic. Using a combination of phylogenetics, structural biology, and enzymology, we show that SIRT4 removes three acyl moieties from lysine residues: methylglutaryl (MG)-, hydroxymethylglutaryl (HMG)-, and 3-methylglutaconyl (MGc)-lysine. The metabolites leading to these post-translational modifications are intermediates in leucine oxidation, and we show a primary role for SIRT4 in controlling this pathway in mice. Furthermore, we find that dysregulated leucine metabolism in SIRT4KO mice leads to elevated basal and stimulated insulin secretion, which progressively develops into glucose intolerance and insulin resistance. These findings identify a robust enzymatic activity for SIRT4, uncover a mechanism controlling branched-chain amino acid flux, and position SIRT4 as a crucial player maintaining insulin secretion and glucose homeostasis during aging.