Cross Validated
2022-09-03 12:54 UTC
By ForceBru
AI-113-20220903-social-media-04e8e5ff
Are discrete mixtures Gauss quadrature-like integral approximations?
I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution. Let $p_{comp}(x)$ be the PDF of a compound distribution given by the integral: $$ p_{comp}(x)=\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta $$ Here we're essentially integrating $p(x, \theta)$ w.r.t. the parameter $\theta$ using the weighting function $p_{param}(\theta)$ which happens to be the PDF of the random parameter $\theta$ . Apply numerical integration like Gauss quadrature: $$ p_{comp}(x) \approx \sum_{k=1}^K w_k p(x,\theta_k) $$ This already looks exactly like the PDF of a mixture, but we need to restrict the weights $w_k$ first: Since numerical integration formulas must integrate polynomials up to some degree exactly, the first condition is that the "mixture" formula must integrate $\theta^0=1$ with weight $p_{param}(\theta)$ exactly: $$\int_{\Theta} p_{param}(\theta)\times 1 d\theta = 1 = \sum_{k=1}^K w_k \times 1$$ The integral equals to one because the weight function $p_{param}(\theta)$ is a PDF. Thus, the weights must sum to one. It's known that all weights $w_k$ of a Gaussian quadrature formula must be positive: $w_k > 0 \quad\forall k$ . Even if we used Newton-Cotes, negative weights seem to be frowned upon since they introduce numerical instabilities. Hence, the weights $w_k$ must be a discrete probability distribution (a probability mass function), mirroring the fact that $p_{param}(\theta)$ is also a prob…
I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution. Let $p_{comp}(x)$ be the PDF of a compound distribution given by the integral: $$ p_{comp}(x)=\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta $$ Here we're essentially integrating $p(x, \theta)$ w.r.t. the parameter $\theta$ using the weighting function $p_{param}(\theta)$ which happens to be the PDF of the random parameter $\theta$ . Apply numerical integration like Gauss quadrature: $$ p_{comp}(x) \approx \sum_{k=1}^K w_k p(x,\theta_k) $$ This already looks exactly like the PDF of a mixture, but we need to restrict the weights $w_k$ first: Since numerical integration formulas must integrate polynomials up to some degree exactly, the first condition is that the "mixture" formula must integrate $\theta^0=1$ with weight $p_{param}(\theta)$ exactly: $$\int_{\Theta} p_{param}(\theta)\times 1 d\theta = 1 = \sum_{k=1}^K w_k \times 1$$ The integral equals to one because the weight function $p_{param}(\theta)$ is a PDF. Thus, the weights must sum to one. It's known that all weights $w_k$ of a Gaussian quadrature formula must be positive: $w_k > 0 \quad\forall k$ . Even if we used Newton-Cotes, negative weights seem to be frowned upon since they introduce numerical instabilities. Hence, the weights $w_k$ must be a discrete probability distribution (a probability mass function), mirroring the fact that $p_{param}(\theta)$ is also a prob…
Full article content could not be extracted automatically. Read the original below.
Source:
Cross Validated
· stats.stackexchange.com