Farm Fresh Insights

Sign in Subscribe

More issues

The Ties That Bound

The previous post described a conservative record-linkage pipeline: accept only high-confidence matches, flag ambiguous cases, and leave the rest unmatched. That is the right approach when false matches are more costly than missed ones, but it means the matched subset is selected on ease of matching, and estimates from it

Join, or Die

Record linkage is the problem of identifying the same entity across two or more tables that lack a shared unique identifier. You have a name, maybe a district, maybe an age, and you need to figure out which row in table A corresponds to which row in table B. When

Recommended Treatment

The effect of a treatment on an outcome depends on three things: the kind of treatment, the kind of person receiving it, and the context in which it is delivered. A ranking change and a button color change work through different mechanisms. A heavy user and a new user will

No Logbook, No Problem: Recovering LATE When You Can't Observe Compliance

Suppose you run a randomized experiment with imperfect compliance. You assigned treatment, measured outcomes, and now you want the LATE. The Wald estimator says divide the ITT by the compliance rate, $\pi = E[D_i \mid Z_i = 1] - E[D_i \mid Z_i = 0]$, and you'

Epic Children, Mostly Sons

Across 450 parent-child entries drawn from 32 epics and 23 civilizations, 85% of children are sons. 63% of couples have zero daughters, while only 6% have zero sons. I classified each entry by how well-documented the family actually is: mythological (gods like Zeus and Brahma), legendary (human figures with no

Regression to the Mean Model

Shuffle your XGBoost training data and you can get a different model. Not different hyperparameters or different features, just a different row order. With tree_method='hist' and multiple threads, the parallel quantile sketch that determines bin boundaries depends on how rows are chunked across workers (see stableboost

The Other Gender Gap

In 152 of 203 countries with available data, more women than men are enrolled in tertiary education. The median Gender Parity Index, the ratio of female to male gross enrollment, is 1.22. The interquartile range is [1.00, 1.35]. Only 51 countries remain below parity. The population-weighted mean

The Wrong Pairameter

In a trust game, the first mover has an endowment and chooses how much to send to the second mover. Whatever is sent gets multiplied. The second mover then decides what fraction of the multiplied amount to return. Sending more creates more total surplus. Returning more determines how that surplus