Connecting many-body magic with quantum criticality with infinite matrix-product states