Orbitronics has attracted significant attention as a platform for information storage and processing that exploits the orbital angular momentum (OAM) of electrons without the need for spin-orbit coupling. However, experimental evaluation of OAM-charge interconversion remains a challenge and the results are often controversial due to the coexistence of bulk and interfacial contributions and the complexity in sample structures. Here, using circularly polarized light pulses and terahertz (THz) polarimetry, we developed a non-contact method to observe the inverse orbital Hall effect as a bulk response within a single material without employing heterostructure samples. In a semiconductor GaN, we directly captured the OAM-to-charge current conversion of holes in the THz frequency range. By analyzing the sharp frequency dependence of the Hall conductivity, we disentangled the competing microscopic mechanisms and revealed the dominant role of extrinsic contributions to the orbital Hall effect in the dc limit. By contrast, the Hall conductivity at THz frequencies above the impurity scattering rate is attributed to the intrinsic Berry-curvature mechanism,allowing a quantitative comparison with the microscopic theory. Furthermore, the ultrafast dynamics of the inverse orbital Hall signal directly revealed sub-picosecond OAM relaxation of holes, comparable to that of phonon-mediated thermalization. The quantitative argument based on theoretical calculations suggested the existence of an even faster decay channel due to momentum redistribution by acoustic phonons, suggesting a sub-nanometer-scale OAM relaxation length. Our results provide comprehensive and crucial insights into OAM transport and establish an ultrafast, contact-free approach for investigating OAM-to-charge conversion.