Lecture Marcello Restelli (marcello.restelli at polimi dot it)
Detailed description of the topics Reinforcement learning deals with solving sequential decision making problems, when no (or minimal) prior information is available. Solving sequential decision making problems means to find their optimal control policies. Using reinforcement-learning algorithms, the optimal policy is learned through the direct interaction between the agent (or controller) and the system to be controlled. The course will introduce the main modeling frameworks, will analyze the most relevant reinforcement-learning techniques, and, finally, some interesting applications of these techniques to real-world domains will be shown. 1) Models * Finite Markov Decision Processes * Continuous Markov Decision Processes * Partially Observable Markov Decision Processes * Semi Markov Decision Processes * Markov Games 2) Algorithms * Value Iteration based algorithms (Q-learning, SARSA, TD(lambda)) * Policy Iteration based algorithms (actor-critic methods, LSPI) * Policy Search algorithms (policy gradient methods and stochastic search techniques) * Exploration techniques (R-MAX, model-based Interval Estimation) * Model-free vs Model-based algorithms * Batch algorithms (Fitted Q-iteration) * Function approximation in Reinforcement Learning algorithms * Hierarchical Learning (options, HAMs, MAX-Q) * Multi-Agent Learning techniques (basic elements) 3) Applications * Autonomic Computing * Robot Control * Water Resources Management * Portfolio Management
Exam The course evaluation can take the form of an oral examination or a project on topics related to the course material.
Schedule The course will start in March and will last for 5 weeks, with two classes of two hours per week.
Bibliography Dimitri P. Bertsekas and John Tsitsiklis, Neuro-Dynamic Programming, Editore: Athena Scientific, Anno edizione: 1996, ISBN: 1-886529-10-8 Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, Editore: MIT Press, Anno edizione: 1998, ISBN: 9780262193986 Csaba Szepesvári, Algorithms for Reinforcement Learning, Editore: Morgan & Claypool, Anno edizione: 2010, ISBN: 1608454924