Sign in Subscribe

Topic

ML/Statistics

A collection of 136 issues

Rank‑Preserving Calibration for Multiclass Probabilities

We often need to adjust a fitted $E[Y|X]$ so that the aggregate $E[Y]$ (or class totals) matches known targets (see here). In binary classification, a single logit shift (or a global temperature) is a monotone transform of the model scores, so if person A had a higher

Generalization Gap in Over‑Parameterized Models

Textbook bias–variance intuition implies that worst‑case test error should eventually rise as model capacity overtakes the sample size because the variance term in the bound grows while the bias term has already bottomed out (see Vapnik and Chervonenkis, 1971; Bartlett and Mendelson, 2002). Those worst‑case guarantees shrink

A Lightweight ALS Solver for Iterative GLS

When generalized least squares spans hundreds of equations, the error covariance becomes the problem. Every loop that updates $\beta$ drags along a $K \times K$ inverse; storing it costs about $O(K^2)$ memory and inverting it costs about $O(K^3)$ time. Even when we model the covariance statistically

Streaming Calibration

Modern applications—from ad platforms calibrating click-through predictions to polling systems incorporating responses to ML algorithms adapting fairness thresholds—share a common challenge: maintaining calibrated weights on live data streams. To address streaming data, we recast raking as a streaming convex optimization problem: minimize the squared error between current weighted

Beam-GD

Gradient descent commits to a single direction at each step based on the local gradient. This myopic approach can be suboptimal when gradients are noisy, local geometry is misleading, or the loss landscape has multiple competing descent directions. The algorithm makes irrevocable decisions based on local information, potentially missing better

Good Enough: Satisficing in Production Machine Learning

Herbert Simon observed that managers rarely chase the global optimum. Instead, they set an aspiration level, a “good enough” performance, and quit searching once they found an option that met it. Simon called this satisficing. That habit makes sense because every decision is a trade‑off. In modeling, the benefit

Boosting Stability: Reducing Instability Under Row Permutation

Shuffle your training data, and XGBoost might give you a different model. Even when you keep features, hyperparameters, and random_state fixed. This behavior violates what most practitioners reasonably expect: that models should be invariant to row permutation. And can lead to silent drift, flaky tests, and spurious alerts. This,

(Better) Split Decisions: Toward A More Structurally Stable CART

Classification and Regression Trees (CART) condense data into one transparent decision tree. Their weakness is structural instability: a small change to the training data can replace an early split, such as “age > 65” with “income > $50 k,” propagating a completely new tree. This instability makes trees unsuitable for

Measuring Tree Stability

Consider training a decision tree to predict customer churn. You split your data in half and build a model on one half. The tree identifies tenure, monthly charges, and contract type as key factors. Satisfied with the results, you rebuild the model on the other random split—but now the

What's Wrong? Adversarial LLM Judges With Their Own Evaluation Criteria

How should we evaluate what large language models say? Metrics like BLEU or exact match don't work well when answers are open-ended or involve reasoning. Human judgments are better—but they're costly and inconsistent. So increasingly, researchers have turned to LLMs themselves to act as evaluators.