I have two independent waves of survey data that I am calibrating back to a known population using iterative proportional fitting (raking).
The population is known with high confidence (all scheduled bus services that a school student could have used), and I am weighting the survey to match the marginal distributions of three variables:
- Service type: School, Regular
- Hour: 7, 8, 14, 15, 16
The survey sample is substantially biased relative to the population because data collection oversampled school services and services at 15:00. Consequently, some combinations that are common in the population have very few observations in the sample.
For example, after raking, a regular service at 7:00 receives a weight of approximately 149. That single observation contributes around 6% of the estimated total trips, which seems to introduce considerable variance and instability.
I understand that trimming weights reduces variance but also introduces bias because the weighted sample no longer exactly reproduces the population margins. My question is about choosing the trimming threshold objectively rather than arbitrarily.
Specifically:
- Is there an accepted statistical framework for selecting a trimming threshold for raking weights?
- Is it reasonable to evaluate a sequence of trimming thresholds (for example, the 99.0th, 99.1st, ..., 99.9th percentiles, where 100% corresponds to no trimming) and compare their estimated bias, variance, and mean squared error (MSE)?
- Are there established methods or references that optimize the bias-variance trade-off for weight trimming?
- Alternatively, are there diagnostic measures such as effective sample size, commonly used to determine an appropriate trimming point?
I am looking for a principled statistical approach for determining the trimming threshold, rather than relying on a subjective choice such as "trim all weights above the 99th percentile."
Also any references to the survey sampling literature would also be appreciated.