Sign in Subscribe

Topic

ML/Statistics

A collection of 130 issues

The Nonscience of Machine Learning

In 2013, Girshick et al. released a paper that described a technique to solve an impossible-sounding problem—classifying each pixel of an image (or semantic segmentation). The technique that they proposed, R-CNN, combines deep learning, selective search, and SVM. It also has all sorts of ad hoc choices, from the

Fairly Certain: Using Uncertainty in Predictions to Diagnose Roots of Unfairness

One conventional definition of group fairness is that the ML algorithms produce predictions where the FPR (or FNR or both) is the same across groups. Fixating on equating FPR etc. can harm the very groups we are trying to help. So it may be useful to rethink how to solve

Optimal Data Collection When Strata and Strata Variances Are Known

With Ken Cor. What’s the least amount of data you need to collect to estimate the population mean with a particular standard error? For the simplest case—estimating the mean of a binomial variable using simple random sampling, a conservative estimate of the variance (p=.5), and a ±3

Equilibrium Fairness: How Fair Algorithms Can Hurt Those They Purport to Help

One definition of a fair algorithm is an algorithm that yields the same FPR across groups (an example of classification parity). To achieve that, we often have to trade in some accuracy. The final model is thus less accurate but fair. There are two concerns with such models: 1. Net

No Stopping: Impact of the Stopping Rule on the Sex Ratio

For social scientists brought up to worry about bias stemming from stopping data collection when results look significant, the fact that a gender-based stopping rule has no impact on the sex ratio seems suspect. So let’s dig deeper. Let there be n families, and let the stopping rule be

ML (O)Ops: What Data To Collect? (part 3)

The first part of the series, “Improving and Deploying On-Device Models With Confidence,” is posted here. The second part, “Keeping Track of Changes,” is posted here. With Atul Dhingra For a broad class of machine learning problems, nitpicking over the neural net architecture is over (see, for instance, here). Instead,

The True Ones: Best Guess of True Proportion of 1s

ML models are generally used to make predictions about individual observations. Sometimes, however, the business decision is based on aggregate data. For example, say a company sells pants and wants to know how many will be returned over a certain period. Say the company has an ML model that predicts

ML (O)Ops! Keeping Track of Changes (Part 2)

The first part of the series, “Improving and Deploying On-Device Models With Confidence”, is posted here. With Atul Dhingra One way to automate classification is to compare new instances to a known list and plug in the majority class of the exact match. For such instance-based learning, you often don’

ML (O)Ops! Improving and Deploying On-Device Models With Confidence (Part 1)

With Atul Dhingra. Part 1 of a multi-part series. It is well known that ML Engineers today spend most of their time doing things that do not have a lot to do with machine learning. They spend time working on technically unsophisticated but important things like deployment of models, keeping

Build Software for the Lay User

Most word processing software helpfully point out grammatical errors and spelling mistakes. Some even autocorrect. And some, like Grammarly, even give style advice. Now, consider software used for business statistics. Say you want to compute the correlation between two vectors: [100, 2000, 300, 400, 500, 600] and [1, 2, 3,