The dust extinction curve is a critical component of many observational programs and an important diagnostic of the physics of the interstellar medium. Here we present new measurements of the dust extinction curve and its variation towards tens of thousands of stars, a hundred-fold larger sample than in existing detailed studies. We use data from the APOGEE spectroscopic survey in combination with ten-band photometry from Pan-STARRS1, 2MASS, and WISE. We find that the extinction curve in the optical through infrared is well characterized by a one-parameter family of curves described by R(V). The extinction curve is more uniform than suggested in past works, with sigma(R(V)) = 0.18, and with less than one percent of sight lines having R(V) > 4. Our data and analysis have revealed two new aspects of Galactic extinction: first, we find significant, wide-area variations in R(V) throughout the Galactic plane. These variations are on scales much larger than individual molecular clouds, posing a challenge to existing paradigms of dust formation and processing: R(V) must be tracing much more than just grain growth in dense molecular regions. Indeed, we find no correlation between R(V) and dust column density up to E(B-V) ~ 2. Second, we discover a strong relationship between R(V) and the far-infrared dust emissivity.