Using a Fractional Factorial Design to Increase Direct Mail Response at Mother Jones Magazine

Experimental design methods have been widely applied to problems in manufacturing, but little attention has been given to applying these tools to service problems. In the world of direct marketing, the traditional approach is called A/B testing and involves changing one factor at a time. In this article we address the problem of improving direct mail response at Mother Jones magazine, employing a 16-run two-level fractional factorial design that tests seven factors simultaneously. We compare this design to other possible design choices, discuss sample size determination, and show how we labeled factors to isolate likely two-factor interactions. We discuss the results and conclusions of our study and the benefits to Mother Jones magazine.