We report a predefined self-organization of gold nanorods (NRs) end-terminated with multiple polymer arms ("pom-poms") in higher-order structures. The assembly of polymer-tethered NRs was controlled by changing the structure of the polymer pom-poms. We show that the variation in the molecular weight of the polymer molecules and their relative location with respect to the long side of the NRs resulted in two competing association modes of the nanorods, that is, their side-by-side and end-to-end assembly, and produced bundles, chains, rings, and bundled chains of the NRs. The superposition of the two variables controlling the organization of NRs allowed us to create a map showing the variation in the longitudinal plasmonic bands of the NRs achieved by their self-assembly.