You are here

Statistical Learning from a Regression Perspective

Richard A. Berk
Publication Date: 
Number of Pages: 
Springer Texts in Statistics
[Reviewed by
Sara Stoudt
, on
See our review of the previous edition of this book.
There may be two cultures of statistical modeling but sometimes a data-driven goal is truly shared. Richard A. Berk’s third edition of Statistical Learning from a Regression Perspective distills both regression and a suite of statistical learning techniques down to being fundamentally interested in the same thing: the conditional distribution of a response as a function of covariates. Under this framing, conceptual connections between machine learning and more traditional statistical modeling that invests in the data-generating process are revealed.
Berk provides a helpful hierarchy of statistical goals: Level I (description), Level II (inference), and Level III (causality) and seeks to partition the assumptions of a variety of statistical learning methods to clarify which assumptions are needed for which level of the hierarchy. Often these assumptions get muddy in practice, and this framework helps delineate what we can and cannot get away with. 
This framework also takes on the elephant in the room, inevitable model mis-specifiation. Guarantees at even the lowest level of the hierarchy evaporate if the model is mis-specified. Berk argues that the relevant target of modeling should be an approximation of the true response surface rather than the true response surface itself. For regression, the goal is then to find the best linear approximation of the true response surface, and for statistical learning methods, this is analogous to a more non-parametric approach where the functional form of the approximation is not set a priori and instead is data-driven. This estimand is more robust to model mis-specification: “one should be making correct inferences to an incorrect model rather than making incorrect inferences to a correct model”.
This book avoids an over-emphasis on the mathematical details, and its strength is the unifying theory of modeling that Berk explains accessibility. It could readily be a textbook for an applications-focused course at the graduate level as each chapter comes with exercises that are focused on the application of the methods and step the reader through trying R implementations of the methods out on data. Examples with accompanying code also appear throughout the chapters which provide a scaffold for getting started, and interested readers, once aware of what tools are available, could then dig in further to the documentation of the suggested packages and functions to proceed further.
Assigning Chapter 1 of this book as supplementary reading would be a strong way to frame a more theoretical introductory statistical learning course for graduate students or a workshop for practitioners with its pragmatic focus (e.g. its constant refrain about the danger of data snooping and ways around it). Reading this chapter as an instructor teaching an undergraduate course on the topic might help ground a class that can often feel like a grab-bag of techniques without a common thread or a natural transition from statistical modeling courses.  
Some new topics like deep learning are included fresh for this edition albeit with a healthy skepticism, with new chapters on neural networks, reinforcement learning, and genetic algorithms. Conformal inference is also introduced as a way to get valid prediction intervals without asymptotic arguments.
I would have liked to see more integration of ethics and more references to that increasingly prominent literature throughout the book rather than just at the end, especially considering that many of the real-world examples used throughout come from criminology. In a future edition, I wonder if this overarching modeling framework would fit fairly naturally with the tidymodels framework in R which has a similar goal of unifying the practice of fitting machine learning models. A translation of some examples using the tidy approach would freshen up some of the current code.
Overall, this book provides a way of thinking about statistical learning problems that builds on the intuition gained from regression modeling and extends that intuition so that the connections in both the algorithms themselves and the approach in which a modeler must take when using the tools are made. Berk’s pragmatic advice will serve a wide audience from practitioners to educators to students.

Sara Stoudt ( is an assistant professor in the Department of Mathematics at Bucknell University. She is interested in applied statistics and the pedagogy of writing in the STEM fields.

See the publisher's website.