We compute the four-loop nf contribution proportional to the quartic Casimir of the QCD cusp anomalous dimension as an expansion for small cusp angle ϕ. This piece is gauge invariant, violates Casimir scaling, and first appears at four loops. It requires the evaluation of genuine non-planar four-loop Feynman integrals. We present results up to O(ϕ4). One motivation for our calculation is to probe a recent conjecture on the all-order structure of the cusp anomalous dimension. As a byproduct we obtain the four-loop HQET wave function anomalous dimension for this color structure.