The World's Largest Research Institute
Dedicated to Discovering How People Make Choices
Dedicated to Discovering How People Make Choices
To be a successful marketer, you must test your ideas. And you need to be confident in the validity of your test results.
In this excerpt from a live session of the University of Florida and MECLABS Institute graduate certificate program, Derrick Jackson, Director of Data Sciences, MECLABS Institute, explains that your test estimation is important because you are trying to foresee what change might happen based on past results. If something radically different happens, it might indicate a validity threat. You should take a close look at the data to determine why there is such a variation from past performance and what you expected.
When engaged in test planning and estimation, you can only be accurate if you carefully consider two factors: (1) How large your test audience is and (2) how large a change you expect between the control and treatment(s) in your test results.
With a bigger change, it would be easier to detect quickly, but if we feel like it's going to be a minute impact, then it would take longer to detect a smaller change.
— Derrick Jackson, Director of Data Sciences, MECLABS Institute
These two factors go together. If your expected test results only show a small change, then you should increase your sample size. But if you foresee a large difference in your test results, then you can get away with a smaller sample size because it is less likely that the difference is due to random chance.
Just don't make the mistake of oversimplifying the process by assuming a large test group ensures accuracy. You also need to consider the size of the change.
And if test estimation is difficult for you, Jackson also gives some tips for those of us who feel that we are just bad at predicting how big of a relative difference we'll get from the test.
This excerpt is part of Session 11 of the MMC 5422 Customer Research and the Fundamentals of Online Testing course, part of our graduate certificate program offered through UF. Click here to learn more about the program.
Related Resources
|
|
Learn how to become a better, more efficient digital marketer Our free digital marketing course contains marketing lessons to study and master the art of online marketing. |
 |
In-person Team Training MECLABS instructors have conducted team-based training on corporate campuses and company headquarters across the nation. |
