Monte-Carlo Expectation Maximization for Decentralized POMDPs

, ,

We address two significant drawbacks of state-of-the-art solvers of decentralized POMDPs (DEC-POMDPs): the reliance on complete knowledge of the model and limited scalability as the complexity of the domain grows. We extend a recently proposed approach for solving DEC-POMDPs via a reduction to the maximum likelihood problem, which in turn can be solved using EM. We introduce a model-free version of this approach that employs Monte-Carlo EM (MCEM). While a naive implementation of MCEM is inadequate in multiagent settings, we introduce several improvements in sampling that produce high-quality results on a variety of DEC-POMDP benchmarks, including large problems with thousands of agents.

» Read More
@inproceedings{WZJijcai13,
 address = {Beijing, China},
 author = {Feng Wu and Shlomo Zilberstein and Nicholas R. Jennings},
 booktitle = {Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI)},
 pages = {397-403},
 title = {Monte-Carlo Expectation Maximization for Decentralized {POMDPs}},
 year = {2013}
}