Follow up to: Not so Prompt: Prompt Optimization as Model Selection The practical problem is to choose a prompt (or a set of prompts) that perform the best on a task in the "real world" while respecting constraints, e.g., the output must parse, latency should stay below

Since the last essay, I have been percolating over whether double descent also means smaller s.e. for HTE. Estimating heterogeneous treatment effects involves two goals: accurate out-of-sample prediction of $\hat{\tau}(x)$ and valid inference for functionals like average treatment effects and policy values. Standard practice constrains model complexity

Many randomized encouragement designs have imperfect compliance, where only a fraction of people comply with their assignment. Examples include phone-bank get-out-the-vote (GOTV) campaigns and draft lotteries like the Vietnam Draft Lottery. In these settings, it is common to use instrumental variable (IV) regression for analysis. Instrumental variables identify the Local

Consider a common scenario: you observe how units (users, devices, firms) interact with items (websites, products, apps), and each item has an unknown binary trait (is it harmful? is it premium content?). You want to understand how these traits relate to unit characteristics through regression, but labeling items is expensive.

An experimentation program is a portfolio problem: at each time $t$, given what we currently know and today's constraints (backlog, traffic budget, test and implementation costs), we must choose a slate of ideas to test, decide how much traffic to allocate, and determine which experiments to continue, stop,

In Basu's circus story (PDF), the owner wants to estimate the weight of fifty elephants and suggests weighing Sambo, who looked “average” three years ago, and reporting $50y$. A statistician counters with a contrived sampling plan: choose Sambo with probability $99/100$ and pick any of the other