The magnetic proximity effect in van der Waals (vdW) heterostructures exerts a significant impact on the electronic and magnetic properties of adjacent materials. Here, we propose vdW heterostructures composed of monolayers ferromagnets (FM) and altermagnets (AM), in which the magnetic proximity effect from the FM induces pronounced non-relativistic valley polarization in the AM. Furthermore, by flipping the magnetization of the FM and the direction of Néel vector of the AM, four independent valley-polarized states can be realized in the FM/AM vdW heterostructures, exhibiting strong magneto-valley coupling. The phenomenon is qualitatively elucidated by a tight-binding model and demonstrated to be generalizable. These findings suggest that FM/AM vdW heterostructures hold potential application in the field of valleytronics-based information storage.