Quantum anomalous Hall effect (QAHE) is a key phenomenon for low power-consumption device applications owing to its dissipationless spin-polarized and quantized current in the absence of an external magnetic field. However, the recorded working temperature of the QAHE is still very low. Here we systematically investigate the magnetic dopants induced modifications from the view points of magnetic, structural and electronic properties and the ultrafast carrier dynamics in a series of V-doped Sb2Te3 samples of composition Sb2-xVxTe3 with x = 0, 0.015 and 0.03. Element specific x-ray magnetic circular dichroism signifies that the ferromagnetism of V-doped Sb2Te3 is governed by the p-d hybridization between the host carrier and the magnetic dopant. Time- and angle-resolved photoemission spectroscopy excited with mid-infrared pulses has revealed that the V impurity induced states underlying the topological surface state (TSS) add scattering channels that significantly shorten the duration of transient surface electrons down to the 100 fs scale. This is in a sharp contrast to the prolonged duration reported for pristine samples though the TSS is located inside the bulk energy gap of the host in either magnetic or non-magnetic cases. It implies the presence of a mobility gap in the bulk energy gap region of the host material that would work toward the robust QAHE. Our findings shed light on the material design for low-energy-consuming device applications.