Sign in Subscribe

Topic

ML/Statistics

A collection of 154 issues

Sampling With Coprimes

Say you want to sample from a sequence of length n. Multiples of a number that is relatively prime to the length of the sequence (n) cover the entire sequence and have the property that the entire sequence is covered before any number is repeated. This is a known result

Beyond Anomaly Detection: Supervised Learning from `Bad' Transactions

Nearly every time you connect to the Internet, multiple servers log a bunch of details about the request. For instance, details about the connecting IPs, the protocol being used, etc. (One popular software for collecting such data is Cisco’s Netflow.) Lots of companies analyze this data in an attempt

Bad Remedies for Bad Science

Lack of reproducibility is a symptom of science in crisis. An eye-catching symptom to be sure, but hardly the only one vying for attention. Recent analyses suggest that nearly two-thirds of the (relevant set of) articles published in prominent political science journals condition on post-treatment variables (see here). Another analysis

When the Cost of Misclassification is Higher for the Customer than for the Business

Consider a bank making decisions about loans. For the bank, making lending decisions optimally means reducing prediction errors*(cost of errors) minus the cost of making predictions (keeping things simple here). The cost of any one particular error, especially the denial of a loan when eligible, is typically small for

The Human and the Machine: Semi-automated approaches to ML

For a class of problems, a combination of algorithms and human input makes for the most optimal solution. For instance, three years ago, software to recreate shredded documents that won the DARPA award used “human[s] [to] verify what the computer was recommending.” The insight is used in character recognition

(No) Value Added Models

This note is in response to some of the points raised in the Agnoff Lecture by Ed Haertel. The lecture makes two big points: 1) Teacher effectiveness ratings based on current Value Added Models are ‘unreliable.’ They are actually much worse than just unreliable; see below. 2) Simulated counterfactuals of

Impact of selection bias in experiments where people treat each other

Selection biases in the participant pool generally have limited impact on inference. One way to estimate population treatment effect from effects estimated using biased samples is to check if treatment effect varies by ‘kinds of people’, and then weight the treatment effect to population marginals. So far so good. When

Sampling on MTurk

In many of the studies that use MTurk, there appears to be little strategy to sampling. A study is posted (and reposted) on MTurk till a particular number of respondents take the study. If the pool of respondents reflects true population proportions, if people arrive in no particular order, and