Sign in Subscribe

Topic

ML/Statistics

A collection of 148 issues

Reducing Errors in Survey Analysis

Analysis of survey data is hard to automate because of the immense variability across survey instruments—different variables, differently coded, and named in ways that often defy even the most fecund imagination. What often replaces complete automation is ad-hoc automation—quickly coded functions, e.g., recoding a variable to lie

Cricket: An Unfairly Random Game?

In many cricket matches, it is claimed that there is a clear advantage to bowling (batting) first. The advantage is pointed to by commentators, and by captains of the competing teams in the pre-toss interview. And sometimes in the post-match interview. The opportunity to bowl or bat first is decided

help(R): matrix indexing

In R, some want to treat matrix like a data frame. It is a bad idea. And no – one cannot use the dollar sign to pick out a column of a matrix. The underlying data structure for a matrix in R is a vector, whereas data.frame object in R

GSS and ANES: Alike Yet Different

The General Social Survey (GSS), run out of National Opinion Research Center at University of Chicago, and American National Election Studies (ANES), which until recently ran out of University of Michigan’s Institute for Social Research, are two preeminent surveys tracking over-time trends in social and political attitudes, beliefs and

The Perils of Balancing Scales

Randomization of scale order (balancing) across respondents is common practice. It is done to ‘cancel’ errors generated by ‘satisficers’ who presumably pick the first option on a scale without regard to content. The practice is assumed to have no impact on the propensity of satisficers to pick the first option,

Join Or Die

How to join datasets correctly is well understood. But we continue to do it badly. Here are notes on how to do fuzzy joins and long to wide transformations starting with some general preprocessing advice. Preprocessing Keys and Tables 1. Remove stuff irrelevant to matching, e.g., font style, spaces,

Q-Tips For R

Distance Between Sounds library(stringdist) > stringdist("ham", "jam", method ="soundex") [1] 1 > stringdist("ham", "banana", method ="soundex") [1] 1 This is pretty bad. Character-wise cosine distance or some other distance based on Soundex codes may be

Flawed Analyses in Deliberative Polls

A Deliberative Poll works as follows: A random sample of people are surveyed. Out of the initial sample, a random subset is invited to deliberate, given balanced briefing materials, randomly assigned to small groups moderated by trained moderators, allowed the opportunity to quiz experts, and in the end surveyed again.