1970 - Ridge Regression (Hoerl & Kennard)

1980 - CHAID Decision Tree (Kass)

1983 - Classification and Regression Trees (Breiman, Friedman et al)

1986 - Generalized Additive Models (Hastie & Tibshirani)

1989 - Thin-Plate Splines (Bookstein)

1991 - Multivariate Adaptive Regression Splines (Friedman)

1992 - Support Vector Machines (Boser, Guyon, Vapnik)

1996 - Lasso (Tibshirani)

1996 - Bagging (Breiman)

2001 - Random Forest (Breiman)

2001 - Gradient Boosting (Friedman)

2005 - Elastic Net (Zou & Hastie)

2005 - RuleFit (Friedman)