MECLABS Methodology
MECLABS is a science lab with a consultancy. We built the first research program to conduct rigorous experiments in the new science of optimization. Our discoveries help leaders optimize the financial performance of their sales and marketing programs.
MECLABS gains this knowledge by utilizing the scientific method to create real-world experiments and then extracting successful principles based on the data generated by this experimentation.
To ensure a consistently high level of scientific rigor in this experimentation, MECLABS research analysts and data scientists employ a uniform test protocol and series of heuristics to help analyze real-world data and create new single factor and multi-factor tests that can reveal further discoveries.
Three Defining Characteristics of the MECLABS Methodology
1. Scientific inquiry
Our research is conducted according to the principles and practices of experimental science. Experiments are designed and conducted in accordance with a larger research plan using a formal test protocol, developed by MECLABS exclusively for its research. Research questions are formed and evaluated as hypotheses, which are tested and the outcomes are statistically validated.
Our research methodology allows us to make the most of any of the industry's widely implemented testing and analytics solutions currently in use (such as Content Experiments from Google, Adobe Target, Google Analytics, Adobe Analytics, ExactTarget Marketing Cloud, Eloqua), whether independently or integrated within our own testing platform (MECData). Using these testing and analytics solutions, we conduct single factor (A/B), multi-factor (multi-variable/MV) and sequential (if unavoidable) tests.
Experimental subjects are generally drawn from a randomly selected sample of real customers or customer prospects, and occasional focus group testing is performed, especially in new product development situations. The full spectrum of online marketing communications is tested, from Web pages to email messages, banner and text ads.
2. Real-world experimentation
MECLABS empirical research focuses on real-world Internet marketing challenges and opportunities. By conducting experiments with actual commercial offerings, we uncover relevant, practical insights that enable our Research Partners to discover what works — for their products, with their target customers, in today’s ever-changing marketplace.
The expertise of MECLABS scientists and analysts is informed by the hundreds of experiments they conduct and participate in each year. Their experiments are informed by these insights, combined with the discipline and wisdom that come from both their formal training in the science of Cognitive Response Optimization and the experience of their prior research.
As technology and culture evolve, MECLABS continues to stay on top of what works best by selecting and collaborating with a small set of organizations capable of responding to and benefiting from increased demand for their products — those with the infrastructure and execution capabilities (e.g., inventory or fulfillment system) that can handle a significant increase in demand.
The findings from these real-world experiments are added to the MECLABS research directory — which we believe to be the most extensive body of experimental research data related to online marketing in existence and is made available at no charge to professional marketers through major experiment briefings in the MECLABS Quarterly Research Digest and through live web clinics.
3. Data-driven principles
MECLABS is a laboratory that uses empirical, relevant data to craft and teach Internet marketing principles. By conducting experiments instead of just relying on so-called “marketer’s intuition,” we are able to discover what really works today. The MECLABS approach transcends “best practices” by discovering and applying principles that help determine when each widely accepted “best practice” works (and when it does not).
The goal of these studies is to discover the most effective way to get prospects to take the desired action, whether it's purchasing a product, starting a subscription, completing a lead form, making a donation, or initiating some other action. We translate the insights garnered from helping actual companies reach tangible objectives into teachable principles that can benefit our worldwide community of almost 80,000 marketers.
Through more than 15 years of extensive experimentation and study, MECLABS research analysts, led by Flint McGlaughlin, have replaced literally hundreds of special case practices with a patented (U.S. Patent Serial No. 8,155,955) unified set of core principles defining and describing the Offer/Response-Optimization process.
These principles are so well defined that they have been taught to hundreds of professional marketers through the MECLABS Training and Certification Program as well as the online marketing lessons in our free digital marketing course that covers marketing psychology, landing page design & conversion rate optimization.
The MECLABS Approach to Experimentation
Experimental science consists of a repeated process of observational inquiry, hypothesis formation, testing, analysis and interpretation. MECLABS has developed and applies a decidedly practical form of this discipline which is centered on an empirically derived model of human behavior, motivation, and decision making consisting of a small set of core principles, or “heuristics.” This model provides proven principles for the planning, design, conduct and interpretation of conversion optimization tests, thereby greatly reducing the time-to-discovery — and to increased demand.
1. Heuristics
Several key elements of the MECLABS methodology are formulated using a heuristic approach. That is, tools to help organize one’s thinking and analysis of a given product offering, marketing message, or experimental design.
For instance, our Conversion Index Analysis method uses the well-known Conversion Optimization Heuristic. This empirically established cognitive framework brings structure and clarity to analysis of the sales conversion process, and guides and simplifies decisions about the prioritization of optimization energy. The Conversion Optimization Heuristic is expressed as follows:
Please note: For the purposes of the MECLABS testing methodology, Friction is defined as a psychological resistance to a given element in the sales or sign-up process. Anxiety is psychological concern stimulated by a given element in the sales or sign-up process.
Another popular heuristic is the Email Messaging Effectiveness Heuristic:
These and other MECLABS heuristics offer powerful transferrable principles that are leveraged in subsequent experiments to continually refine and improve our effectiveness toward advancing our discipline, and bringing value to our Research Partners.
2. Discipline-specific data analysis
MECLABS testing methodology relies extensively on data analysis in three key areas: understanding target customers, identifying optimization opportunities and assessing test outcomes.
Creating the optimal message and process for the target customer is the essence of optimization. MECLABS employs quantitative methods to assemble, organize and study all available objective evidence across a variety of sources in pursuit of a robust Customer Theory. Because there can be as many different kinds of customers as there are products and use-cases, analysis of demographic, behavioral and operational data can reveal valuable insights by both identifying meaningful differences among the ideal customer segments and informing the Customer Theory for each important persona.
Today’s digital marketing professional is faced with a perpetual parade of competing metrics software and testing tools — each with its own strengths and limitations, owing to its particular approach and nomenclature. The MECLABS approach to online marketing optimization is science-centric, and consequently can be robustly applied using any commercial-grade metrics or testing solution — even across different platforms.
While all of the current commercial grade testing and metrics systems help you with how to test, the greatest obstacle to breakthrough optimization gains is establishing what to test. The result is, marketers get seemingly significant results, masking the fact that testing a different page, or a different step in a process, would have produced something far more meaningful and profitable.The MECLABS integrated approach to experimental analytics combines the organizing power of its optimization heuristics with the clarifying power of corresponding, purpose-specific mathematical and statistical methods, to navigate the critical path toward the optimal sales funnel.
3. Single factor and multi-factor testing
Once the marketing challenge has been evaluated and described, using a set of time-tested principles and sound analytical methods, MECLABS research analysts employ its Conversion Optimization meta-theory to develop hypotheses about what must be changed to achieve better funnel performance, and design real-world experiments to test them. In accordance with the research objectives, as well as the available tools and traffic, experiments are designed to maximize the value of each test to achieve the greatest long-term value in the shortest period of time. Typically, that includes a mix of single factor (i.e., A/B) and multi-factor (i.e., multivariable or MV) tests.
4. Test Protocol
The test protocol is a software-based tool developed by MECLABS to guide and facilitate the experimental testing process. It is the document of record for each test, and the repository of test protocols is an essential component of the MECLABS knowledge base.
The design of the Test Protocol is continually refined through ongoing experimentation. The composition, structure, and sequence of the test protocol are designed to guide the process and enforce the standards and discipline of scientific experimentation aimed at hypothesis testing.
Depending on the key metric, a statistical test planning tool is used to estimate the sample size required and to project test duration.
As the test data is gathered, it is accumulated in the test protocol and the computations refreshed periodically to test for validity. Depending upon the experiment design, a single test sequence may involve an iterative approach to identifying and pruning out under-performing treatments while adaptively introducing new variations as informed by the prior recursions. This approach can be used to accelerate the pace of discovery with minimal risk of missing an optimal solution.
Once the optimal experimental treatment is identified — or the design sample size is exceeded and the test is deemed inconclusive (i.e., based on the data received, a winner cannot be chosen in an acceptable time frame) — the complete data set is analyzed in the context of the original set of hypotheses to evaluate whether they are supported by the experimental data or whether they must be rejected. While being right is usually gratifying, we often learn more from the tests that prove us wrong than those that suggest that we had it right — for therein lie the seeds of new discovery.
The last step of every test sequence is interpretation and recommendations for further research. Since the test hypotheses, and the treatment designs, are informed by the then-current body of MECLABS optimization principles, this step ensures that MECLABS continually stays on top of changes in “what really works” as markets, tools and cultures emerge and evolve; and the principles thereby remain “evergreen.”
Let the Institute Work for You: Free diagnostic and related research
MECLABS provides curated resources and services ranging from free web clinics to customized optimization services — all based on customer-first science that has helped increase revenue for organizations ranging from Fortune 50 to emerging businesses.
Please provide your contact information below. A MECLABS analyst will respond within 24 hours and ask a few questions about your business to determine how we can best serve you. They will also immediately provide valuable content, at no charge, that addresses your specific challenges.
