Sign in Subscribe

Topic

Statistics

A collection of 114 issues

The True Ones: Best Guess of True Proportion of 1s

ML models are generally used to make predictions about individual observations. Sometimes, however, the business decision is based on aggregate data. For example, say a company sells pants and wants to know how many will be returned over a certain period. Say the company has an ML model that predicts

ML (O)Ops! Keeping Track of Changes (Part 2)

The first part of the series, “Improving and Deploying On-Device Models With Confidence”, is posted here. With Atul Dhingra One way to automate classification is to compare new instances to a known list and plug in the majority class of the exact match. For such instance-based learning, you often don’

ML (O)Ops! Improving and Deploying On-Device Models With Confidence (Part 1)

With Atul Dhingra. Part 1 of a multi-part series. It is well known that ML Engineers today spend most of their time doing things that do not have a lot to do with machine learning. They spend time working on technically unsophisticated but important things like deployment of models, keeping

Build Software for the Lay User

Most word processing software helpfully point out grammatical errors and spelling mistakes. Some even autocorrect. And some, like Grammarly, even give style advice. Now consider software used for business statistics. Say you want to compute the correlation between two vectors: [100, 2000, 300, 400, 500, 600] and [1, 2, 3,

Superhuman: Can ML Beat Human-Level Performance in Supervised Models?

A supervised model cannot do better than its labels. (I revisit this point later.) So the trick is to make labels as good as you can. The errors in labels stem from three sources: 1. Lack of Effort: More effort people spend labeling something, presumably the more accurate it will

Too Much Churn: Estimating Customer Churn

A new paper uses financial transaction data to estimate customer churn in consumer-facing companies. The paper defines churn as follows: There are three concerns with the definition: 1. The definition doesn’t make clear what is the normalizing constant for calculating the share. Given that the value “can vary between

94.5% Certain That Covid Vaccine Will Be Less Than 94.5% Effective

“On Sunday, an independent monitoring board broke the code to examine 95 infections that were recorded starting two weeks after volunteers’ second dose — and discovered all but five illnesses occurred in participants who got the placebo.” Moderna Says Its COVID-19 Vaccine Is 94.5% Effective In Early Tests The data

Fat Or Not: Toward 'Proper Training of DL Models'

A new paper introduces a DL model to enable ‘computer aided diagnosis of obesity.’ Some concerns: 1. Better baselines: BMI is easy to calculate and it would be useful to compare the results to BMI. 2. Incorrect statement: The authors write: “the data partition in all the image sets are

Not So Robust: The Limitations of "Doubly Robust" ATE Estimators

Doubly Robust (DR) estimators of ATE are all the rage. One popular DR estimator is Robins’ Augmented IPW (AIPW). The reason why Robins’ AIPW estimator is called doubly robust is that if either your IPW model or your y ~ x model is correctly specified, you get ATE. Great! Calling something

Instrumental Music: When It Rains, It Pours

In a new paper, Jon Mellon reviews 185 papers that use weather as an instrument and finds that researchers have linked 137 variables to weather. You can read it as each paper needing to contend with 136 violations of the exclusion restriction, but the situation is likely less dire. For