Prostate Cancer (PCa) is the most commonly diagnosed malignancy and second leading cause of cancer-related mortality in men. With the use of next generation sequencing and proteomic platforms, new biomarkers are constantly being developed to both improve diagnostic sensitivity and specificity and help stratify patients into different risk groups for optimal management. In recent years, it has become well accepted that altered glycosylation is a hallmark of cancer progression and that the glycan structures resulting from these mechanisms show tremendous promise as both diagnostic and prognostic biomarkers. In PCa, a wide range of structural alterations to glycans have been reported such as variations in sialylation and fucosylation, changes in branching, altered levels of Lewis and sialyl Lewis antigens, as well as the emergence of high mannose "cryptic" structures, which may be immunogenic and therapeutically relevant. Furthermore, aberrant expression of galectins, glycolipids, and proteoglycans have also been reported and associated with PCa cell survival and metastasis. In this review, we discuss the findings from various studies that have explored altered N- and O-linked glycosylation in PCa tissue and body fluids. We further discuss changes in O-GlcNAcylation as well as altered expression of galectins and glycoconjugates and their effects on PCa progression. Finally, we emphasize the clinical utility and potential impact of exploiting glycans as both biomarkers and therapeutic targets to improve our ability to diagnose clinically relevant tumors as well as expand treatment options for patients with advanced disease.