Sign in Subscribe

Gaurav

https://gsood.com

Sample This: Sampling Randomly from the Streets

Say you want to learn about the average number of potholes per unit paved street in a city. To estimate that quantity, the following sampling plan can be employed: 1. Get all the streets in a city from Google Maps or OSM 2. Starting from one end of the street,

Clustering and then Classifying: Improving Prediction for Crudely-labeled and Mislabeled Data

Mislabeled and crudely labeled data are common problems in data science. Supervised prediction of such data is expectedly to yield poor results—coefficients are biased, and accuracy with regard to the true label is poor. One solution to the problem is to hand-code labels of an `adequate’ sample and infer

The Missing Plot

Datasets often contain missing values. And often enough—at least in social science data—values are missing systematically. So how do we visualize missing values? After all, they are missing. Some analysts simply list-wise delete points with missing values. Others impute, replacing missing values with mean or median. Yet others

About 85% Problematic: The Trouble With Human-In-The-Loop ML Systems

MIT researchers recently unveiled a system that combines machine learning with input from users to ‘predict 85% of the attacks.’ Each day, the system winnows down millions of rows to a few hundred atypical data points and passes these points on to ‘human experts’ who then label the few hundred

Interpeting Clusters and 'Outliers' from Clustering Algorithms

Assume that the data from the dominant data generating process are structured so that they occupy a few small portions of a high-dimensional space. Say we use a hard partition clustering algorithm to learn the structure of the data. And say that it does—learn the structure. Anything that lies

Sampling With Coprimes

Say you want to sample from a sequence of length n. Multiples of a number that is relatively prime to the length of the sequence (n) cover the entire sequence and have the property that the entire sequence is covered before any number is repeated. This is a known result

Clarifai(ng) the Future of Clarifai

Clarifai is a promising AI start-up. In a short(ish) time, it has made major progress on an important problem. And it is rapidly rolling out products with lots of business potential. But there are still some things that it could do. As I understand it, the base version of