A comparative analysis of the genetic diversity of ancient and modern sheep can shed light on the origin of these animals and their distribution as well as help to evaluate the role of humans at each formation stage of different sheep breeds. Here we isolated ancient DNA and performed sequencing of the mitochondrial DNA D-loop from 17 sheep bone remains (~4000–1000 years old) found in the archaeological complexes in the south of Altai (Western Siberia). The length of the sequences obtained ranged between 318 and 586 bp. The haplotype diversity and nucleotide diversity were 0.801 ± 0.081 and 0.0096 ± 0.0014 respectively. The average number of nucleotide differences was ~3.1. Nucleotide sequence analysis revealed that 15 specimens were nested within previously described A,B,C,D and E lineages and that two specimens had a basal position relative to the rest of the analyzed samples. A relatively high diversity of sheep haplotypes, including the presence of two basal haplotypes, indicates that the Altai region may have been a transport route of human migration. Further ancient DNA analysis of other specimens and deeper genome sequencing of samples with novel haplotypes is needed to better understand the demographic history of sheep in Southern Siberia.