Sex is a biological variable that contributes to individual variability in brain structure and behavior. Neuroimaging studies of population-based samples have identified normative differences in brain structure between males and females, many of which are exacerbated in psychiatric and neurological conditions. Still, sex differences in MRI outcomes are understudied, particularly in clinical samples with known sex differences in disease risk, prevalence, and expression of clinical symptoms. Here we review the existing literature on sex differences in adult brain structure in normative samples and in 14 distinct psychiatric and neurological disorders. We discuss commonalities and sources of variance in study designs, analysis procedures, disease subtype effects, and the impact of these factors on MRI interpretation. Lastly, we identify key problems in the neuroimaging literature on sex differences and offer potential recommendations to address current barriers and optimize rigor and reproducibility. In particular, we emphasize the importance of large-scale neuroimaging initiatives such as the Enhancing NeuroImaging Genetics through Meta-Analyses consortium, the UK Biobank, Human Connectome Project, and others to provide unprecedented power to evaluate sex-specific phenotypes in major brain diseases.