Conflicting reports exist on the ground state of the intermetallic compound CeRhSn₂. This can be rooted in the sawtooth-like arrangement of two inequivalent Ce sites in the unit cell, which suggests potential geometric magnetic frustration. To resolve, we conducted a comprehensive study on high-quality single crystals of CeRhSn₂ by means of magnetization (M), specific heat (Cₚ/T), and resistivity (ρ). The system exhibits strong magnetic anisotropy, confirming the b-axis as the easy magnetic axis. We establish three successive transitions, an AFM order at T_N = 3.65K, a first-order FM order at T_C = 1.7K and final transition, at T = 1.5K. The transition temperatures are highly field-directional dependent: in a magnetic field, the lowest transition is immediately suppressed while H ∥ b rapidly merges T_C and T_N into a single second-order transition. Conversely, H∥ c suppresses the FM order and reduces T_N. Additional ab initio calculations affirm the FM ground state of CeRhSn₂. The observation of an enhancement of the Sommerfeld coefficient (γ = 76.5mJ/mol·K²) may arise from geometric frustration, but it is most consistently attributed to weak Kondo hybridization as frustration cannot be conclusively established through our data.