Over the past few decades, ultracold atomic gases have emerged as an exciting playground for the curious minded to probe many-body quantum physics in a technique known as quantum simulation. Here, the motion of neutral atoms in crystalline-like optical potentials enable experimentalists to realize the Hubbard model, which predicts a wide variety of condensed matter phenomena. A particularly enticing avenue of exploration is to go beyond the ground band and incorporate excited states of the lattice potential. This extension greatly enhances the range of phenomena that we can study in the laboratory. In this talk, I shall first introduce the fundamental concepts of ultracold atoms and quantum simulation in optical lattices. Then I will discuss a few illustrative experiments probing transport and interaction phenomena and how they are enriched by considering excited bands. This will conclude with a discussion of how we can apply such systems to investigate non-equilibrium quantum physics and quantum thermalization. In parallel, I will highlight the important role of students in the research laboratory and my vision for next-generation cold atom experiments.