Several Phase II and III clinical trials have demonstrated that immunotherapy can induce objective responses in otherwise refractory malignancies in tumors outside the central nervous system. In large part, effector T cells mediate much of the antitumor efficacy in these trials, and potent antitumor T cells can be generated through vaccination, immune checkpoint blockade, adoptive transfer, and genetic manipulation. However, activated T cells must still traffic to, infiltrate, and persist within tumor in order to mediate tumor lysis. These requirements for efficacy pose unique challenges for brain tumor immunotherapy, due to specific anatomical barriers and populations of specialized immune cells within the central nervous system that function to constrain immunity. Both autoimmune and infectious diseases of the central nervous system provide a wealth of information on how T cells can successfully migrate to the central nervous system and then engender sustained immune responses. In this review, we will examine the commonalities in the efferent arm of immunity to the brain for autoimmunity, infection, and tumor immunotherapy to identify key factors underlying potent immune responses.