Spectra carry key aspects of the physical state of galaxies across the observable universe. Yet a realistic generative model of galaxy spectra has been largely elusive, likely because of intrinsic variability and evolution in the physical processes we seek to understand. Recently, Portillo et al. (2020) demonstrated that SDSS spectra can efficiently be modeled by an autoencoder with ~10 latent variables, but only after artificially de-redshifting all spectra. We extend this prior work by including a redshift estimation step in the encoder and an explicit redshifting transformation in the decoder. Doing so relegates the neural net to only represent the restframe features in its latent space. We also utilize a novel architecture for more efficient spectral compression. Taken together, we can produce data-driven models of galaxy spectra as well as neural redshift estimation across all available redshifts and even combine multiple spectroscopic surveys.