We give a detailed treatment of electromagnetic signals generated by gravitational waves (GWs) in resonant cavity experiments. Our investigation corrects and builds upon previous studies by carefully accounting for the gauge dependence of relevant quantities. We work in a preferred frame for the laboratory, the proper detector frame, and show how to resum short-wavelength effects to provide analytic results that are exact for GWs of arbitrary wavelength. This formalism allows us to firmly establish that, contrary to previous claims, cavity experiments such as ADMX, HAYSTAC, CAPP and ORGAN designed for the detection of axion dark matter only need to reanalyze existing data to search for high-frequency GWs with strains as small as h~10^-22-10^-21. We also argue that directional detection is possible in principle using readout of multiple cavity modes. Further improvements in sensitivity are expected with cutting-edge advances in superconducting cavity technology.