Bayesian Inference Forgetting. (arXiv:2101.06417v1 [cs.LG])

The right to be forgotten has been legislated in many countries but the
enforcement in machine learning would cause unbearable costs: companies may
need to delete whole models trained from massive resources because of single
individual requests. Existing works propose to remove the influence of the
requested datums on the learned models via its influence function which is no
longer naturally well-defined in Bayesian inference. To address this problem,
this paper proposes a {it Bayesian inference forgetting} (BIF) framework to
extend the applicable domain to Bayesian inference. In the BIF framework, we
develop forgetting algorithms for variational inference and Markov chain Monte
Carlo. We show that our algorithms can provably remove the influence of single
datums on the learned models. Theoretical analysis demonstrates that our
algorithms have guaranteed generalizability. Experiments of Gaussian mixture
models on the synthetic dataset and Bayesian neural networks on the
Fashion-MNIST dataset verify the feasibility of our methods. The source code
package is available at url{https://github.com/fshp971/BIF}.

Source: https://arxiv.org/abs/2101.06417

webmaster

Related post