Bandit tutorial slides and update on book

This website has been quiet for some time, but we have not given up on bandits just yet.

First up, we recently gave a short tutorial at AAAI that covered the basics of finite-armed stochastic bandits and stochastic linear bandits. The slides are now available (part1, part2).

We mentioned at some point that these notes would become a book. We’re happy to say this project is coming close to completion. We hope in the next month or so to publish a reasonable quality draft.

Of course the book contains all the content in the blog in a polished and extended form. There are also many new chapters. Some highlights are: combinatorial bandits, non-stationary bandits, ranking, Bayesian methods (including Thompson sampling) and pure exploration. We also have two chapters that peek beyond the world of bandits at partial monitoring and learning in Markov decision processes.

Once the draft is complete we would love to have your feedback.