Hole spin qubits based on semiconductor quantum dots are promising for building future large-scale quantum computers owing to their all-electrical manipulation. However, abundant physical mechanisms of valence band holes lead to anisotropic qubit properties. There is an opportunity to prolong the coherence time and achieve high-fidelity qubit manipulations. Here, we report a single-hole spin qubit in a planar germanium quantum dot and investigate its anisotropic susceptibility to charge noise under an in-plane magnetic field. By correlating the longitudinal spin-electric susceptibility with qubit coherence and control performance, we identify an optimal operating point where the sensitivity to charge noise is minimized. We find that optimizing the magnetic-field orientation reduces the spin-electric susceptibility, resulting in a five-fold enhancement of the Hahn-echo coherence time and a nearly tenfold suppression of control infidelity. At the optimal operating point, gate set tomography demonstrates the maximum gate fidelity of 99.82 %. Our finding enhances the prospects of hole spin qubits for scalable quantum information processing.