Nonlinear spectroscopy is widely used to probe ultrafast dynamics in matter. Signals of higher nonlinear orders encode valuable many-body dynamics but only recently have become individually retrievable. However, their physical interpretation in terms of double-sided Feynman diagrams is cumbersome and non-intuitive. Here, we establish an alternative framework that connects them to the underlying excitation conditions and many-body interactions. This approach predicts two classes of dynamic nonlinearities, due either to direct many-body interactions or to saturable kinetic channels. The method provides a direct physical interpretation of the retrieved signals, explaining their sign, magnitude, and dynamics. It also enables model-free reconstruction of the dynamics of a sub-ensemble of particles starting with a given number of excitons, which we demonstrate to be especially insightful. We validate the framework using colloidal CdSe/CdS core/shell quantum dots, which enable us to model nonlinearities of both classes in isolation and also combined.