Hubble's discovery of the expansion of the Universe inexorably linked the study of the smallest phenomena (particle physics) to that of the largest (cosmology). Observations of the Universe in its infancy reveal a striking confirmation that the same physics that explains phenomena on Earth today shaped the Universe in the distant past. However, this very consistency has posed some of the most exciting fundamental and as-yet-unanswered questions in Physics. In this colloquium, I will recount both the story of matter and the Universe and how this history demonstrates both great triumphs and failures of the Standard Model to describe our world. Finally, I will highlight the outstanding mysteries which constitute some of the most exciting opportunities for discovery in theory, experiment and observation.