Hexagonal boron nitride (h-BN) epilayers have been recognized as a promising material for applications in solid-state neutron detectors. However, the highest detection efficiency of 58% attained so far for 1 mm2 detectors fabricated from 50 μm thick B-10 enriched h-BN films still falls short of the expected theoretical value of 64%. This is due to the less than perfect charge collection efficiency. In this work, we have fabricated and analyzed the photocurrent-voltage characteristics of 11 h-BN neutron detectors. The dependence of the charge collection efficiency (ηc) on the charge carrier mobility-lifetime (μτ) product of the bulk trapping parameter reveals that ηc is nearly perfect at a bias voltage of 200 V if we neglect the effects of surface recombination. Our results have clearly demonstrated that the surface recombination of charge carriers is the dominant factor that prevents a further enhancement in the charge collection efficiency in our current detectors. The surface recombination field ES (=s/μ), defined as the ratio of the surface recombination velocity (s) to the carrier mobility (μ) of holes, was found to have a linear relationship with that of electrons and has a magnitude of the order of 104 V/cm. The present study indicates that it is critical to reduce ES in h-BN in order to further push the charge collection and hence the total detection efficiency of h-BN neutron detectors to 100%.