This article explores the relationship between gossip and sexuality in early nineteenth-century France, through consideration of the biography of Astolphe de Custine and the novels of the so-called querelle d'Olivier: Madame de Duras's Olivier, ou le secret (c. 1821); Henri de Latouche's Olivier (1826); and Stendhal's Armance (1827). These three romans à clé were inspired, at least in part, by two events in Custine's life: his breaking of an engagement to Mme de Duras's daughter Clara in 1818, and his ‘outing’ as a homosexual in 1824, when he was brutally beaten for having propositioned a guardsman. Both events occasioned outbursts of gossip within Restoration high society, and this article considers the Olivier novels as part of, and as responses to, that gossip: though motivated by personal or political animosity, the novels equally frame an ambivalent critique of the process whereby gossip enforces social norms, notably in matters of sexuality. The two later novels, composed in the wake of Custine's humiliation of 1824, even seem to anticipate the structure of the modern closet; but the article concludes by underlining both the historical specificity and the broad metaphorical valence of the texts' evocation of sexual secrets.