In this paper, we describe an adaptive optimal PI controller for high-precision low-temperature experiments. We used a modified LQR algorithm to obtain the optimal parameters for the PI controller. Since the plant is nonlinear, a gain scheduling method is applied to modify the optimal parameters under different operating conditions. The simulation results show that this controller has good transient response, disturbance rejection ability, and robustness. Furthermore, the controller can accommodate a variety of first-order nonlinear systems. At last, we discuss the design of the optimal PID controller for a class of second order system using the modified LQR algorithm.