We report a comprehensive study of the low-temperature magnetic properties of the insulating rare-earth triangular magnet CsErSe₂. We uncover a field-induced level crossing at the crystal electric field (CEF) ground state, which gives rise to a first order phase transition as well as to distinctive magnetic properties. This crossing is identified by the accurate determination of the single-ion Hamiltonian, the reliability of which is substantially enhanced by field-dependent optical spectroscopy that directly tracks the Zeeman splittings of Kramers doublets. We also observe spontaneous antiferromagnetic ordering in CsErSe₂ below T_N ≈ 110 mK, and resolve the corresponding magnetic structure using elastic neutron scattering. We discuss how the rich magnetic behavior of CsErSe₂ arises from the interplay of non-trivial field-dependent single-ion physics and spontaneous ordering, and highlight the implications of these results for understanding magnetic phenomena across a wide range of insulating rare-earth magnets.