i should write an article on what economists do when we don't have the data we need, meaning the exercises we do because it's good practice or because reviewers will ask. this after we already considered changing the question :)
something like "problem -> exercise":
1. the variable exists but you can't observe it -> sensitivity analysis (Oster, Rosenbaum)
2. it exists somewhere, just not in your data -> proxies, external calibration
3. part of the sample is missing, not at random -> Lee bounds, Horowitz-Manski, selection models
4. you have it, but badly measured -> EIV corrections, reliability ratios, Monte Carlo
5. you measure quantity but need quality/prices -> hedonics, residual inference from identities
6. the counterfactual doesn't exist (lol) -> the design builds it, placebos audit it, honest DiD prices its failure
7. the original inputs are lost/proprietary -> GOOD LUCK (jk: partial reconstruction, scenario grids, email the authors!!)
8. the data is at the wrong level -> ecological-inference bounds, shift-share
9. the definitions changed -> cry, then crosswalks, bridge tables, splicing
10. too few observations -> randomization inference, synthetic control placebos, shrinkage
11. datasets you can't link -> probabilistic linkage, two-sample IV, mismatch bounds
12. suppressed/top-coded on purpose -> interval regression, Pareto-tail imputation
13. the phenomenon is newer than the data -> nowcasting, triangulate unofficial sources
14. the data describes the wrong population -> reweight, bound the unobservables (Andrews–Oster)
15. your variable answers a nearby question, not yours (patents =/= innovation) -> convergent validation, or change the estimand
16. one unit's treatment leaks onto others -> exposure mappings, spillover-radius sensitivity
and it's all really just four moves:
BOUND IT: give up the point, report the honest set
STRESS IT: how wrong must the assumption be to kill the result?
REBUILD IT: buy the point back with structure and side data
SIMULATE IT: build the missing world, watch your estimator in it
To whomsoever is saddled with talking me out of blowing up the planet in the future:
My supervillain origin story is definitely my LaTeX tables refusing to compile after 6 hrs of re-editing, while giving me no information about the underlying error.
Hope this helps.
@kreepkroop Privilege of one kind is used to fight oppression of another. Pointing that out is not an inherently divisive rhetoric! systems exist that some can take adv of and others systematically cannot, to keep themselves safe! But yes, never truly feel safe. Only a matter of degree ig