Statistical Rules of Thumb

One day not long ago, I was in a debate with a psychologist concerning the use of one- versus two-sided hypothesis tests. The psychologist insisted that there was never a need to use a two-sided test when the researcher was only interested in detecting differences in one direction. I disagreed, arguing that doing so simply increases your chance of crossing the “magic” 5% threshold (or any other magic threshold one wishes to use). At the end of the day, we agreed to disagree and she went ahead with her one-sided tests.

It is in arguments such as these where Statistical Rules of Thumb can come in quite handy. Rule-of-thumb 1.12 clearly states that two-sided tests should generally be used and one-sided tests are discouraged. While I doubt this would have changed my colleague’s mind, I can still take a small measure of satisfaction in knowing that my rule of thumb made it into Gerald van Belle’s classic book. Much ground is covered in this useful reference resulting in a book that has something for just about everyone.

For each topic he discusses, van Belle provides the reader with a brief introduction and rule of thumb, an illustration which provides some technical background, the basis for the rule, and discussion with more examples. These rules cover a wide range of statistical methods and procedures. For example, he discusses calculation of sample sizes required when using the coefficient of determination (page 32), how many observations one should have to construct a confidence interval (page 20), whether to use a simple random sample or a more complicated sampling scheme (page 95), and whether or not to use the delta method or bootstrapping when one is estimating complex relationships within data (page 191). The chapters are organized nicely so that one can easily find any potential rules available for any particular topic.

Chapter 1 provides basic rules such as when the +/–2 standard error rule for forming a 95% confidence interval is robust. Chapter 2 covers rules for calculating required sample sizes in many setting. Chapter 3 gives rules for analyzing data from observational studies (which is new to this edition), and chapter 4 provides guidelines for analyzing covariation in data (such as warning against the restricted range problem in regression).

Chapters 5, 6, and 7 cover areas in which Dr. van Belle is an expert — rules for environmental studies, epidemiology, and evidence-based medicine (also new to this edition) are covered. He rounds out the final three chapters by providing insights on design and analysis of experiments, methods to display data, and topics concerning statistical consultation.

For the applied researcher who does much of her or his own data analysis, this book is a must-have. Even the applied statistician would benefit from owning a copy of this collection. It is certain that some “rules” will be new, and the descriptions in the text can come in quite handy when one is trying to explain a concept to a non-statistician. In short, this collection of “rules” is highly recommended.

Liam O'Brien is assistant professor of statistics at Colby College in Waterville, ME.