Posterior-Aided Regularization for Likelihood-Free Inference. (arXiv:2102.07770v1 [cs.LG])

The recent development of likelihood-free inference aims training a flexible
density estimator for the target posterior with a set of input-output pairs
from simulation. Given the diversity of simulation structures, it is difficult
to find a single unified inference method for each simulation model. This paper
proposes a universally applicable regularization technique, called
Posterior-Aided Regularization (PAR), which is applicable to learning the
density estimator, regardless of the model structure. Particularly, PAR solves
the mode collapse problem that arises as the output dimension of the simulation
increases. PAR resolves this posterior mode degeneracy through a mixture of 1)
the reverse KL divergence with the mode seeking property; and 2) the mutual
information for the high quality representation on likelihood. Because of the
estimation intractability of PAR, we provide a unified estimation method of PAR
to estimate both reverse KL term and mutual information term with a single
neural network. Afterwards, we theoretically prove the asymptotic convergence
of the regularized optimal solution to the unregularized optimal solution as
the regularization magnitude converges to zero. Additionally, we empirically
show that past sequential neural likelihood inferences in conjunction with PAR
present the statistically significant gains on diverse simulation tasks.



Related post