Back

G-2026-80

Policy-consistent inverse optimal control for discrete-time nonlinear stochastic systems

, , and

BibTeX reference

Inverse Optimal Control (IOC) seeks to recover an unknown cost from expert demonstrations, and it provides a systematic way of modeling experts' decision mechanisms while considering the prior information of the dynamics and the cost functions. Nevertheless, existing IOC methods lack consistency guarantees under noisy and nonlinear settings, and hence cannot guarantee policy-consistent recovery. In this paper, we consider a discrete-time nonlinear stochastic system, and it is controlled by an optimal policy that minimizes the expectation of a discounted cumulative cost function across an infinite time-horizon. In particular, the cost function takes the form of a linear combination of a priori known feature functions. In this setting, we first adopt Lasserre's reformulation (Hernández-Lerma and Lasserre, 1996) of the forward problem with occupancy measure. Next, we propose the infinite dimensional IOC algorithm and further approximate it with Lagrange interpolating polynomials, which results in a convex, finite-dimensional sum-of-squares optimization. Moreover, the estimator is shown to be asymptotically and statistically consistent in optimality with respect to the demonstrated expert trajectories. The subsequent policy estimator is also shown to be asymptotically and statistically consistent in policy recovery. Finally, we validate the theoretical results and illustrate the performance of our method with numerical experiments. In addition, the robustness and generalizability performance of the proposed IOC algorithm are also illustrated.

, 26 pages

Research Axes

Research application

Document

G2680.pdf (400 KB)