Counterfactual Generation with Identifiability Guarantees

Hanqi Yan, Lingjing Kong, Lin Gui, Yujie Chi, Eric Xing, Yulan He, Kun Zhang

Research output: Contribution to conference typesPaperpeer-review

192 Downloads (Pure)


Counterfactual generation lies at the core of various machine learning tasks, including image translation and controllable text generation. This generation process usually requires the identification of the disentangled latent representations, such as content and style, that underlie the observed data. However, it becomes more challenging when faced with a scarcity of paired data and labeling information. Existing disentangled methods crucially rely on oversimplified assumptions, such as assuming independent content and style variables, to identify the latent variables, even though such assumptions may not hold for complex data distributions. For instance, food reviews tend to involve words like “tasty”, whereas movie reviews commonly contain words such as “thrilling” for the same positive sentiment. This problem is exacerbated when data are sampled from multiple domains since the dependence between content and style may vary significantly over domains. In this work, we tackle the domain-varying dependence between the content and the style variables inherent in the counterfactual generation task. We provide identification guarantees for such latent-variable models by leveraging the relative sparsity of the influences from different latent variables. Our theoretical insights enable the development of a doMain AdapTive counTerfactual gEneration model, called (MATTE). Our theoretically grounded framework achieves state-of-the-art performance in unsupervised style transfer tasks, where neither paired data nor style labels are utilized, across four large-scale datasets.
Original languageEnglish
Publication statusPublished - 2023
Event37th International Conference on Neural Information Processing Systems, NeurIPS 2023 - New Orleans, LA, New Orleans, United States
Duration: 10 Dec 202316 Dec 2023
Conference number: 37


Conference37th International Conference on Neural Information Processing Systems, NeurIPS 2023
Abbreviated titleNeurIPS 2023
Country/TerritoryUnited States
CityNew Orleans
Internet address

Cite this