In The Media

Analyzing Your Advertising
by Michael E. Duffy

Source: MarketingProfs, April 2004

Albert Einstein once said that insanity is doing the same thing over and over again, and expecting a different outcome. Put another way, given the same inputs to the same processes, it's a pretty good bet that the same thing is gonna happen 99 out of 100 times.

If you're not happy with the outcomes you're getting in your business, what are you doing to change things?

Advertising is one of those things that businesses do over and over again, frequently without any measurable success. Yellow Page ads, newspaper and magazine advertising, even websites are common business expenses with uncertain value. As John Wanamaker, inventor of the department store, put it: "I know that half of my advertising is wasted. I just don't know which half."

One of the tenets of direct mail advertising is: "test, test, test." This idea has been carried over to the Web, where it is possible for a website to present a visitor with different versions of a Web page, and their response rate can be directly measured.

The problem with any sort of testing is trying to decide whether you've found the optimum combination of elements such as the headline, the copy, the offer itself and the graphic design. For example, just two choices for each of these four elements means there are 16 possible combinations to test. Add more elements, or more choices, and the sheer number of different combinations which must be tested to find the optimal advertisement or Web page becomes enormous.

It turns out that the whole problem of optimization when a large number of variables are involved is something that turns up in other activities, such as building cars. In the late 1940s, a Japanese engineer named Dr. Genichi Taguchi developed a method (the eponymous "Taguchi method") to reduce a problem involving thousands or millions of combinations into a relatively small number of experiments which could be conducted simultaneously. His methodology was brought to the United States by researchers at Bell Laboratories and made its way into the engineering departments of large companies. Ford used Taguchi methods to design the original Taurus, which was heralded as Detroit's answer to the consumer-orientation of Japanese automakers and became the #1 selling automobile in the U.S.

For some reason, Taguchi's methods remained relatively unknown outside of big companies, in spite of the fact that there is a global business in teaching Taguchi methods (a search on Amazon.com returns 913 books). Enter Dr. James Kowalick (www.kowalick.com). Kowalick was introduced to Taguchi methods in 1985 when he was the director of corporate engineering for Aerojet General. Subsequently, he was involved in teaching others to use Taguchi methods through the Renaissance Institute at Cal Tech.

Kowalick's breakthrough realization, however, was that Taguchi's approach to optimization could be applied to almost everything, including cookie recipes and (more profitably) advertising. Kowalick's methodology breaks down into a five step process. First, you select the "success factors." For example, the subject line, message text and contact information are so important in e-mail marketing that they are mandatory. The remaining factors can be chosen from offers (such as "15% off" vs. "free sample"), sender, image, color, fonts or one of four "custom" factors (e.g., what day of the week you send or place the ad).

For each of the success factors, you then create alternative versions (called "options"). For example, one possibility is to have six alternative versions for an e-mail subject line and three options for each of the remaining factors. Each alternative version must have a nickname.

As an example, six choices of subject line and three options for each of the remaining six factors means there are 4,374 possible combinations. The methodology generates 18 test messages which cover the same territory (according to Taguchi's methodology and apparently born out by actual results).

It's then up to you to create the 18 ads based on those descriptions, mail them out and track the results. You'll also need a mailing list with at least 5,400 names on it since the methodology requires 18 randomly selected groups (one for each test message), each containing 300-1,500 recipients. Kowalick recommends doing two distinct mailings to different lists to improve results.

Once you've mailed and tabulated the results, you enter the response rate for each of the 18 (or 36) test groupings. Based on the various response rates, the methodology derives an "optimum" ad, i.e., the best option to use for each of the success factors. Some success factors may be labeled "weak," which means there was no appreciable difference in the options, and so the option may be chosen based on cost or convenience (you still must use one of the options originally created, of course). According to Kowalick, mailing this optimized message to a large list should generate the maximum possible response rate. Kowalick's website highlights businesses which have used the technique successfully: an insurance agency, a mortgage broker, a furniture company and a maker of Celtic jewelry.

If your business spends serious dollars on advertising (or advises those who do), you should definitely take a deeper look at Kowalick's methodology.

Back