Does the soft-constraint converge to rigid-constraint

textitle: “Does the soft-constraint converge to rigid-constraint?” author: “Yuling Yao” date: “10/21/2018” output: html_documentMotivationsRigid constraintSoft constraintAdding a Jacobian?Generative vs Embedding, or change-of-variable vs conditioningPrior-Prior conflict?Conclusion title: “Does the soft-constraint converge to rigid-constraint?”
author: “Yuling Yao”
date: “10/21/2018”
output: html_document tl;nr.
Motivations I have often seen the recommendation of soft-constraints. The main reason is because a bayesian will rarely prefer point mass. Apart from that, I am now curious whether the current implementation of these two in Stan are essentially equivalent, in the limit case.
Let me frame the problem more religiously. Consider the parameter space θ∈Θ=Rd\theta \in \Theta = R^dθ∈Θ=Rd , and a transformed parameters
z=ξ(θ),z∈Rm z=\xi(\theta),\quad z \in R^m z=ξ(θ),z∈Rm
where $\xi : R^d \to R^m, d>m $ is a smooth function. We also need some regularization on ξ\xiξ to make…

A Decoupling Perspective of Projective Inference

I have recenetly been reading a textbook on decoupling. Decoupling literally means "from dependence toindependence". I initially though the book can help me understand some properties of self-normalized importance sampling,but it turns out too mathematical to be any related to my work, but sometimes I also read NY times, which is probably even less relevant.

Anyway, I read the great paper “Projective Inference in High-dimensional Problems: Prediction and Feature Selection” (by Juho Piironen, Markus Paasiniemi, andAki Vehtari) this afternoon. The paper is wonderful, though it is hard to find any paper from Aki that is not wonderful. I realize some of their results might also be demonstrated in the language of decoupling.

Warning: No, this blog is not going to give a more natural or more straightforward introduction. I am writing this blog to justify why I read that textbook at all.

Lower variance from Rao-Blackwellization

The motivation behind projective inference can be easi…