Multi-agent Reinforcement Learning for Groundwater Markets*
Session Number
1
Advisor(s)
Igor Cialenco, Illinois Institute of Technology
Location
A117
Discipline
Computer Science
Start Date
15-4-2026 10:15 AM
End Date
15-4-2026 11:00 AM
Abstract
Groundwater overextraction poses a critical threat to agricultural and environmental sustainability worldwide. Recent work developed by Cialenco and Ludkovski (2025) introduces a stochastic game-theoretic model of groundwater trading markets, in which farmers optimize their reward through crop production, water trading, and intertemporal water banking, all subject to a stochastic aquifer recharge. While the resulting Nash equilibrium is fully characterized in the single-period setting, the multi-period game remains computationally intractable for classical naive methods as the number of agents or the number of time periods increase. In this work, we develop a multi-agent reinforcement learning (MARL) framework which approximates the multi-period Nash equilibrium of the groundwater market model. We implement a Multi-Agent Soft Actor-Critic (MA-SAC) algorithm under Centralized Training with Decentralized Execution (CTDE). Each agent learns a strategy over consumption and banking decisions while an established centralized critic stabilizes training by observing the joint state. We present numerical experiments for two-agent and three-agent markets. Our results characterize the equilibrium banking behavior of agents both in the short term, with agents' banking patterns in response to stochastic recharge; and in the long term, with what general proportion agents allocate to banking.
Multi-agent Reinforcement Learning for Groundwater Markets*
A117
Groundwater overextraction poses a critical threat to agricultural and environmental sustainability worldwide. Recent work developed by Cialenco and Ludkovski (2025) introduces a stochastic game-theoretic model of groundwater trading markets, in which farmers optimize their reward through crop production, water trading, and intertemporal water banking, all subject to a stochastic aquifer recharge. While the resulting Nash equilibrium is fully characterized in the single-period setting, the multi-period game remains computationally intractable for classical naive methods as the number of agents or the number of time periods increase. In this work, we develop a multi-agent reinforcement learning (MARL) framework which approximates the multi-period Nash equilibrium of the groundwater market model. We implement a Multi-Agent Soft Actor-Critic (MA-SAC) algorithm under Centralized Training with Decentralized Execution (CTDE). Each agent learns a strategy over consumption and banking decisions while an established centralized critic stabilizes training by observing the joint state. We present numerical experiments for two-agent and three-agent markets. Our results characterize the equilibrium banking behavior of agents both in the short term, with agents' banking patterns in response to stochastic recharge; and in the long term, with what general proportion agents allocate to banking.