Large crystals BaGa4Se7 (BGSe), promising as a nonlinear optical converters in the mid-IR, were grown and investigated using methods of optical spectroscopy. The crystals are transparent in the range 0.465 ÷ 22 μm. The Tauc analysis showed that BGSe is a direct bandgap dielectric with Eg = 2.73 and 2.91 eV at 300 and 80 K, respectively, with dEg/dT = −8.07 × 10−4 eV/K. Vibrational modes are observed in the range 150–350 cm−1 in the Raman spectra: Thus, two-phonon absorption determines the transparency edge in the mid-IR. Main extended defects (the (001) microtwins and point defects (impurity centers such as Ga–O and different native centers) are revealed. BGSe crystals are photosensitive in the range of 1.5–4.0 eV. Luminescence in BGSe is quenched by two orders of intensity as temperature increases from 77 to 300 K. The possibility of controlling the luminescence parameters by post-growth annealing in a suitable atmosphere demonstrated. Pyroelectric luminescence in the 100–440 K range confirms the noncentrosymmetric structure of this crystal.