Under study is the diversity of metric balls in connected finite ordinary graphs considered as a metric space with the usual shortest-path metric. We investigate the structure of graphs in which all balls of fixed radius i are distinct for each i less than the diameter of the graph. Let us refer to such graphs as graphs with full diversity of balls. For these graphs, we establish some properties connected with the existence of bottlenecks and find out the configuration of blocks in the graph. Using the obtained properties, we describe the tree-like structure graphs with full diversity of balls.