Polymer thermoelectric materials have attracted increasing attention and exhibited great potential in green energy conversion due to their inexpensive to process, lightweight, mechanical flexibility, and intrinsic low thermal conductivity in the past decades. Integration of polymers with nanofillers has been one main approach to tailor the thermoelectric properties of polymers, which not only combines the properties of each component but also displays interesting transport phenomenon induced by the interfaces between polymers and nanofillers. In this review, we discussed the thermoelectric properties of polymer composites, especially for the conducting polymer based composites, including polymer/inorganic, polymer/carbon nanofillers, polymer/polymer hybrids. The fabrication and structure-dependent thermoelectric properties were discussed. In addition, polymer composites based thermoelectric devices were also systematically addressed, and potential issues in the devices fabrication were also analyzed. Regarding the future research opportunities, some visionary discussion was also provided.