Neuropsychiatric disease has polygenic determinants but is often precipitated by environmental pressures, including adverse perinatal events. However, the way in which genetic vulnerability and early-life adversity interact remains obscure. We hypothesised that the extreme environmental stress of prematurity would promote neuroanatomic abnormality in individuals genetically vulnerable to psychiatric disorders. In 194 unrelated infants (104 males, 90 females), born before 33 weeks of gestation (mean gestational age 29.7 weeks), we combined Magnetic Resonance Imaging with a polygenic risk score (PRS) for five psychiatric pathologies to test the prediction that: deep grey matter abnormalities frequently seen in preterm infants are associated with increased polygenic risk for psychiatric illness. The variance explained by the PRS in the relative volumes of four deep grey matter structures (caudate nucleus, thalamus, subthalamic nucleus and lentiform nucleus) was estimated using linear regression both for the full, mixed ancestral, cohort and a subsample of European infants. Psychiatric PRS was negatively associated with lentiform volume in the full cohort (β = −0.24, p = 8 × 10−4) and a European subsample (β = −0.24, p = 8 × 10−3). Genetic variants associated with neuropsychiatric disease increase vulnerability to abnormal lentiform development after perinatal stress and are associated with neuroanatomic changes in the perinatal period.