Dynamical selection of Nash equilibria using reinforcement learning: Emergence of heterogeneous mixed equilibria

Robin Nicole*, Peter Sollich

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)
113 Downloads (Pure)

Abstract

We study the distribution of strategies in a large game that models how agents choose among different double auction markets. We classify the possible mean field Nash equilibria, which include potentially segregated states where an agent population can split into subpopulations adopting different strategies. As the game is aggregative, the actual equilibrium strategy distributions remain undetermined, however. We therefore compare with the results of a reinforcement learning dynamics inspired by Experience-Weighted Attraction (EWA) learning, which at long times leads to Nash equilibria in the appropriate limits of large intensity of choice, low noise (long agent memory) and perfect imputation of missing scores (fictitious play). The learning dynamics breaks the indeterminacy of the Nash equilibria. Non-trivially, depending on how the relevant limits are taken, more than one type of equilibrium can be selected. These include the standard homogeneous mixed and heterogeneous pure states, but also heterogeneous mixed states where different agents play different strategies that are not all pure. The analysis of the reinforcement learning involves Fokker-Planck modeling combined with large deviation methods. The theoretical results are confirmed by multi-agent simulations.

Original languageEnglish
Article numbere0196577
Number of pages21
JournalPLoS ONE
Volume13
Issue number7
Early online date9 Jul 2018
DOIs
Publication statusPublished - 2018

Fingerprint

Dive into the research topics of 'Dynamical selection of Nash equilibria using reinforcement learning: Emergence of heterogeneous mixed equilibria'. Together they form a unique fingerprint.

Cite this