Topological insulators are novel quantum materials with metallic surface transport but insulating bulk behavior. Often, topological insulators are dominated by bulk contributions due to defect induced bulk carriers, making it difficult to isolate the more interesting surface transport characteristics. Here, we report the synthesis and characterization of nanosheets of a topological insulator Bi2Se3 with variable Sb-doping levels to control the electron carrier density and surface transport behavior. (Bi1-xSbx)2Se3 thin films of thickness less than 10 nm are prepared by epitaxial growth on mica substrates in a vapor transport setup. The introduction of Sb in Bi2Se3 effectively suppresses the room temperature electron density from ∼4 × 1013 cm-2 in pure Bi2Se3 (x = 0) to ∼2 × 1012 cm-2 in (Bi 1-xSbx)2Se3 at x ∼ 0.15, while maintaining the metallic transport behavior. At x ≳ ∼0.20, a metal-insulator transition (MIT) is observed, indicating that the system has transformed into an insulator in which the metallic surface conduction is blocked. In agreement with the observed MIT, Raman spectroscopy reveals the emergence of vibrational modes arising from Sb-Sb and Sb-Se bonds at high Sb concentrations, confirming the appearance of the Sb2Se3 crystal structure in the sample. These results suggest that nanostructured chalcogenide films with controlled doping can be a tunable platform for fundamental studies and electronic applications of topological insulator systems.