One of the main problems in comparative studying animal behavior is searching for an adequate mathematical method for evaluating the similarities and differences between behavioral patterns. This study aims to propose a new tool to evaluate ethological differences between species. We developed the new compression-based method for the homogeneity testing and classification to investigate hunting behavior of small mammals. A distinction of this approach is that it belongs to the framework of mathematical statistics and allows one to compare the structural characteristics of any texts in pairwise comparisons. To validate a new method, we compared the hunting behaviors of different species of small mammals as ethological "texts." To do this, we coded behavioral elements with different letters. We then tested the hypothesis whether the behavioral sequences of different species as "texts" are generated either by a single source or by different ones. Based on association coefficients obtained from pairwise comparisons, we built a new classification of types of hunting behaviors, which brought a unique insight into how particular elements of hunting behavior in rodents changed and evolved. We suggest the compression-based method for homogeneity testing as a relevant tool for behavioral and evolutionary analysis.