Sign in Subscribe

Topic

ML/Statistics

A collection of 151 issues

Sample This

What do single shot evaluations of MT (replace it with anything else) samples (vis-a-vis census figures) tell us? I am afraid very little. Inference rests upon knowledge of the data (here – respondent) generating process. Without a model of the data generating process, all such research reverts to modest tautology – sample

Representativeness Heuristic, Base Rates, and Bayes

From the Introduction of their edited volume: Tversky and Kahneman used the following experiment for testing the ‘representativeness heuristic’ – Subjects are shown a brief personality description of several individuals, sampled at random from 100 professionals – engineers and lawyers. Subjects are asked to assess whether the description is of an engineer

Comparing Datasets and Reporting Only Non-Duplicated Rows

The following is in response to a question on the R-Help list. Consider two datasets: reported <- structure(list(Product = structure(c(1L, 1L, 1L, 1L, 2L, 2L, 3L, 4L, 5L, 5L), .Label = c("Cocoa", "Coffee C", "GC", "Sugar No 11"

Correcting for Differential Measurement Error in Experiments

Differential measurement error across control and treatment groups or in a within-subjects experiment, pre- and post-treatment measurement waves, can vitiate estimates of treatment effect. One reason for differential measurement error in surveys is differential motivation. For instance, if participants in the control group (pre-treatment survey) are less motivated to respond

Against Proxy Variables

Lacking direct measures of the theoretical variable of interest, some rely on “proxy variables.” For instance, some have used years of education as a proxy for cognitive ability. However, using “proxy variables” can be problematic for the following reasons: (1) proxy variables may not track the theoretical variable of interest

Recoding Variables Reliably and Systematically

Survey datasets typically require a fair bit of repetitive recoding of variables. Reducing errors in recoding can be done by writing functions carefully (see some tips here) and automating and systematizing naming, and application of the recode function (which can be customized) – fromlist <- c("var1", "

Working With Modestly Large Datasets in R

Even modestly large (< 1 GB) datasets can quickly overwhelm modern personal computers. Working with such datasets in R can be still more frustrating because of how R uses memory. Here are a few tips on how to work with modestly large datasets in R. Setting Memory Limits On Windows,

Reconceptualizing the Effect of the Deliberative Poll

Deliberative Poll proceeds as follows: Respondents are surveyed, provided ‘balanced’ briefing materials, randomly assigned to moderated small group discussions, allowed the opportunity to quiz experts or politicians in plenary sessions, and re-interviewed at the end. The “effect” is conceptualized as the average Post–Pre across all participants. The effect of

Weighting to Multiple Datasets

Say there are two datasets—one that carries attitudinal variables, and demographic variables (dataset 1), and another that carries just demographic variables (dataset 2). Also assume that Dataset 2 is the more accurate and larger dataset for demographics (e.g. CPS). Our goal is to weight a dataset (dataset 3)