Webinar link
Nº du webinaire : 910 7928 6959
Code secret : VISS

Restless bandits are a class of sequential resource allocation problems concerned with allocating one or more resources among several alternative processes where the evolution of the process depends on the resource allocated to them. In 1988, Whittle proposed an index heuristic for restless bandit problems which has emerged as a popular solution approach due to its simplicity and strong empirical performance. The Whittle index heuristic is applicable if the model satisfies a technical condition known as indexability. We present two general sufficient conditions for indexability and identify simpler to verify refinements of these conditions for fully-observable models and a class of indexable models for the partially-observable ones. We show that a generalization of the adaptive greedy algorithm computes the Whittle index for all indexable restless bandits. Finally, we discuss a learning problem in restless bandits when the dynamics of all processes are unknown initially. The learning algorithm uses a Thompson sampling approach to estimate the unknown parameters and uses estimated Whittle index to choose actions. We provide an upper-bound on the regret with respect to an oracle who knows the true parameters.