Evaluating Regression Models
(link)
Some Ideas on How to Think About Vectors
I’d like to share a quick video on some ideas on how to think about and teach about vectors in a data science context. (link)
The Purpose of our Data Science Chalk Talk Series
I’d like to share an introduction to my data science chalk talk series (video link, series link)
What is Statistics?
I recently shared a bit of the history of The Science of Data Analysis. I thought I would follow that up with a quick chalk talk titled “What is Statistics?” (link)
The Science of Data Analysis
I am re-reading from the great statistician John W. Tukey’s paper: Tukey, John W. “The Future of Data Analysis.” Ann. Math. Statist. 33 (1962), no. 1, pp. 1–67. doi:10.1214/aoms/1177704711. https://projecteuclid.org/euclid.aoms/1177704711 I’ve taken the liberty of pulling out some quotes that are very relevant to the usual “data science is not […]
New Free Mini-Lecture: Simpson’s Paradox in A/B Testing
I’d like to share a new fee mini-lecture on avoiding Simpson’s Paradox when analyzing A/B test results.
A Single Parameter Family Characterizing Probability Model Performance
Introduction We’ve been writing on the distribution density shapes expected for probability models in ROC (receiver operator characteristic) plots, double density plots, and normal/logit-normal densities frameworks. I thought I would re-approach the issue with a specific family of examples.
An Example of a Calibrated Model that is not Fully Calibrated
Our group has written a lot on calibration of models and even conditional calibration of models. In our last note we mentioned the possibility of “fully calibrated models.” This note is an example of a probability model that is calibrated in the traditional sense, but not fully calibrated in a […]
The Double Density Plot Contains a Lot of Useful Information
The double density plot contains a lot of useful information. This is a plot that shows the distribution of a continuous model score, conditioned on the binary categorical outcome to be predicted. As with most density plots: the y-axis is an abstract quantity called density picked such that the area […]
Your Lopsided Model is Out to Get You
For classification problems I argue one of the biggest steps you can take to improve the quality and utility of your models is to prefer models that return scores or return probabilities instead of classification rules. Doing this also opens a second large opportunity for improvement: working with your domain […]
