Time-dependent density functional theory, though exact in theory, is in practice applied in an adiabatic approximation using exchange-correlation functionals with only local temporal dependence. Simultaneously, the exact correlation potential formally depends on, among other quantities, the time-history of electron densities. Here we develop a framework to learn neural network models of correlation functionals that feature explicit memory-dependence. Our framework includes two different approaches, both linked through their use of adjoint-based optimization. One approach decouples the learning of the functional from inversion to find ground truth values of the correlation potential. The other approach learns the functional directly without requiring inversion. We apply these methods to modeling the electron dynamics of two-electron systems in two spatial dimensions. In both cases, our methods yield correlation functionals with low test set propagation error, outperforming standard local density and generalized gradient approximation functionals by one to two orders of magnitude. Overall, our framework points at one strategy to move beyond the adiabatic approximation and develop memory-dependent correlation functionals that yield accurate propagation for excited state and/or non-equilibrium dynamics.