We study finite sample guarantees for transport based mutual information estimation. Mutual information can be estimated by first learning the conditional and marginal densities and then evaluating their log-density ratio by Monte Carlo. Given training samples, we use triangular transport maps to learn these densities and then compute the mutual information estimate on an independent batch of evaluation samples. For conditional density estimation using triangular transport maps, we establish non-asymptotic guarantees over realizable Hölder classes by viewing the conditional density as an ordinary density with respect to the product of the law of the conditioning variable and Lebesgue measure. This allows standard density estimation techniques to be applied directly to the conditional density estimation. These results yield finite sample guarantees for the resulting mutual information estimator and characterize how a fixed sample budget should be allocated between training and evaluation in order to minimize the mean squared error. Numerical experiments support the predicted training and evaluation error rates and illustrate the resulting sample allocation tradeoff.