Results of a search for the pair production of photon-jets - collimated groupings of photons - in the ATLAS detector at the Large Hadron Collider are reported. Highly collimated photon-jets can arise from the decay of new, highly boosted particles that can decay to multiple photons collimated enough to be identified in the electromagnetic calorimeter as a single, photonlike energy cluster. Data from proton-proton collisions at a center-of-mass energy of 13 TeV, corresponding to an integrated luminosity of 36.7 fb-1, were collected in 2015 and 2016. Candidate photon-jet pair production events are selected from those containing two reconstructed photons using a set of identification criteria much less stringent than that typically used for the selection of photons, with additional criteria applied to provide improved sensitivity to photon-jets. Narrow excesses in the reconstructed diphoton mass spectra are searched for. The observed mass spectra are consistent with the Standard Model background expectation. The results are interpreted in the context of a model containing a new, high-mass scalar particle with narrow width, X, that decays into pairs of photon-jets via new, light particles, a. Upper limits are placed on the cross section times the product of branching ratios σ×B(X→aa)×B(a→γγ)2 for 200 GeV<mX<2 TeV and for ranges of ma from a lower mass of 100 MeV up to between 2 and 10 GeV, depending upon mX. Upper limits are also placed on σ×B(X→aa)×B(a→3π0)2 for the same range of mX and for ranges of ma from a lower mass of 500 MeV up to between 2 and 10 GeV.