Optically driven chiral phonons can induce transient magnetism in non-magnetic crystals, yet the microscopic relationship between phonon angular momentum and the resulting magnetic response remains unresolved. Here we distinguish the roles of pseudo-angular momentum and mechanical angular momentum in tetragonal α-TeO2. Combining phase-resolved terahertz spectroscopy with magneto-optical Kerr measurements (MOKE), we reconstruct the motion of a doubly degenerate phonon and continuously control its angular momentum. The MOKE response follows the sign and magnitude of phonon angular momentum as well as its dynamics, whereas non-rotating phonons, including those with finite pseudo-angular momentum, produce no detectable magnetic signal. The MOKE rotation corresponds to an effective magnetic field of approximately 14 mT, nearly seven orders of magnitude above the estimate for spatially uniform ionic currents. Moreover, the Kerr response scales linearly with the emitted terahertz field, rather than quadratically as predicted by the uniform-current model. A localized Raman-excitation model involving Te lone-pair states accounts for this scaling and yields a comparable field magnitude with the MOKE data when electronic enhancement is included. These findings identify mechanical lattice rotation as the dominant dynamical variable for the observed magnetic response and suggest that localized Raman coupling contributes to its unexpectedly large magnitude.