Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems. (arXiv:2108.11959v1 [cs.LG])

Autoregressive exogenous (ARX) systems are the general class of input-output
dynamical systems used for modeling stochastic linear dynamical systems (LDS)
including partially observable LDS such as LQG systems. In this work, we study
the problem of system identification and adaptive control of unknown ARX
systems. We provide finite-time learning guarantees for the ARX systems under
both open-loop and closed-loop data collection. Using these guarantees, we
design adaptive control algorithms for unknown ARX systems with arbitrary
strongly convex or convex quadratic regulating costs. Under strongly convex
cost functions, we design an adaptive control algorithm based on online
gradient descent to design and update the controllers that are constructed via
a convex controller reparametrization. We show that our algorithm has
$tilde{mathcal{O}}(sqrt{T})$ regret via explore and commit approach and if
the model estimates are updated in epochs using closed-loop data collection, it
attains the optimal regret of $text{polylog}(T)$ after $T$ time-steps of
interaction. For the case of convex quadratic cost functions, we propose an
adaptive control algorithm that deploys the optimism in the face of uncertainty
principle to design the controller. In this setting, we show that the explore
and commit approach has a regret upper bound of $tilde{mathcal{O}}(T^{2/3})$,
and the adaptive control with continuous model estimate updates attains
$tilde{mathcal{O}}(sqrt{T})$ regret after $T$ time-steps.