Layered Learning in Multiagent Systems: A Winning Approach to Robotic Soccer

This book looks at multiagent systems that consist of teams of autonomous agents acting in real-time, noisy, collaborative, and adversarial environments. The book makes four main contributions to the fields of machine learning and multiagent systems.First, it describes an architecture within which a flexible team structure allows member agents to decompose a task into flexible roles and to switch roles while acting. Second, it presents layered learning, a general-purpose machine-learning method for complex domains in which learning a mapping directly from agents' sensors to their actuators is intractable with existing machine-learning methods.

Third, the book introduces a new multiagent reinforcement learning algorithm—team-partitioned,opaque-transition reinforcement learning (TPOT-RL)—designed for domains in which agents cannot necessarily observe the state-changes caused by other agents' actions. The final contribution is a fully functioning multiagent system that incorporates learning in a real-time, noisy domain with teammates and adversaries—a computer-simulated robotic soccer team.Peter Stone's work is the basis for the CMUnited Robotic Soccer Team, which has dominated recent RoboCup competitions. RoboCup not only helps roboticists to prove their theories in a realistic situation, but has drawn considerable public and professional attention to the field of intelligent robotics. The CMUnited team won the1999 Stockholm simulator competition, outscoring its opponents by the rather impressive cumulative score of 110-0.

This book looks at multiagent systems that consist of teams of autonomous agents acting in real-time, noisy, collaborative, and adversarial environments.

Electronic Review of Computer Books - Jack J. Woehr

Layered Learning in Multiagent Systems, by Peter Stone, is a computer-science diary of a voyage the author and his associates took to the frontiers of machine learning and machine interaction with the immanent, rather than virtual plane of existence.

The subtitle of the book is "A Winning Approach to Robotic Soccer." Stone, a soccer player and fan since childhood, did his Ph.D. research participating in perfecting robotic soccer and fielding Carnegie Mellon University CMUnited teams in both simulator competition and competitions of real-world robot teams. The theoretical exploration, the decisions that were made, and the resulting competitions are the subject matter of the book.

Psychologists, epistemologists, and game theorists might appreciate Layered Learning in Multiagent Systems as much as computerists will. How one goes about teaching machines to think, strategize collectively, and react is freighted with opportunities for minute experimentation beyond those available to clinical researchers. It's as if the robot team was a clutch of motorized ugly ducklings with Stone acting the role of mother duck, helping to teach them to socialize and forage.

In short ...

"the local pass-evaluation information, while helpful, is not enough on which to base a decision. The agent needs to learn strategic information which depends on the behaviors of teammates and oppononents and which can only be measured by its team's long-term success at achieving its collective goal against an opponent team in a real game."

This bleeding-edge speculation upon mechanical reasoning finds echoes in Seneca of Rome and in Plutarch of Chaeronea, who wrote 19 centuries ago in his essay "On the Use of Reason by So-Called Irrational Animals":

"Animals' intelligence refuses to accommodate any expertise which has no point...and doesn't regard skills as matters to be imported from others ... rather it generates the necessary skills on the spot out of itself as if they were native and natural products."

and reminds us why that which we call "science" in the present day was called of old "natural philosphy".

The speculation, in the present instance, is well grounded in practice. The CMUnited team won the 1999 Simulator competition in Stockholm with a combined score over the other teams of 110-0. CMUnited-98 in the small-robot league of RoboCup-98 won 4 of 5 games with a total score of 25-6. One long-term goal of the competitions is to have humanoid robots playing soccer by 2050.

Although firmly grounded in theory, Layered Learning in Multiagent Systems refrains from notational extravagances and requires of you no more than basic statistics, trigonometry, and logic calculus. As the author notes, the book's innovations are "designed primarily for real-time, noisy, collaborative and adversarial domains." A domain which sorta resembles real life, and certainly one encountered in the real-time control projects many of us have worked on. If autonomous robotics and adaptive machine intelligence are your bag, this attractive volume is one of the more readable efforts.

Peter Stone is Assistant Professor of Computer Sciences, Alfred P. Sloan Research Fellow, and Director of the Learning Agents Group, University of Texas at Austin. He is the recipient of the International Joint Conference on Artificial Intelligence (IJCAI) 2007 Computers and Thought Award.