Articular cartilage consists of chondrocytes and two major components, a collagen-rich framework and highly abundant proteoglycans. Most prior studies defining the zonal distribution of cartilage have extracted proteins with guanidine-HCl. However, an unextracted collagen-rich residual is left after extraction. In addition, the high abundance of anionic polysaccharide molecules extracted from cartilage adversely affects the chromatographic separation. In this study, we established a method for removing chondrocytes from cartilage sections with minimal extracellular matrix protein loss. The addition of surfactant to guanidine-HCl extraction buffer improved protein solubility. Ultrafiltration removed interference from polysaccharides and salts. Almost four-times more collagen peptides were extracted by the in situ trypsin digestion method. However, as expected, proteoglycans were more abundant within the guanidine-HCl extraction. These different methods were used to extract cartilage sections from different cartilage layers (superficial, intermediate, and deep), joint types (knee and hip), and disease states (healthy and osteoarthritic), and the extractions were evaluated by quantitative and qualitative proteomic analyses. The results of this study led to the identifications of the potential biomarkers of osteoarthritis (OA), OA progression, and the joint specific biomarkers.