01. Is your system a software
or a consulting service?
It is a consulting service.
02. How do you pick the factors
for multivariate testing?
We use the 80/20 rule: we do a multivariate test of the 20%
percent of the factors that represent 80% of the success and
we use split-testing for the less influential factors.
03. Is the list considered a
Yes, but a different kind of variable than the components
being optimized. Lists must be tested separately from the
04. How viable is your system
to markets under 15,000 prospects?
Depends on the medium. For example, an email optimization
needs a minimum of 10,000 names, a direct mail optimization
needs a minimum of 50,000 names, a landing page optimization
requires a minimum of 1,500 visitors per day.
05. What if I do not want to
test so many variables? How can I test fewer variables or
use fewer tests?
We only test the most critical variables, applying the 80/20
principle described in Question # 2. For the non-critical
(20% of the influence) we use split testing (A/B, A/B/C, etc).
06. Can multivariate testing be used for Pay-Per-Click (PPC) ads?
Absolutely. A PPC ad has 4 variables (headline, description
1, description 2 and URL) so it is possible to test 3 variations
of each with 9 tests to find the optimum ad.
07. How do you measure success
- actions or sales?
Both. In our projects we track responses and conversions.
08. How much variance is there
among media format? For example, if you test 9 variations
of a direct mail piece,
how close would the results match if you tested the same 9
creatives in e-mail, or landing pages?
We found that there is a 60-70% similarity.
09. How well does multivariate
testing work for landing page optimization?
Very well. In a way landing pages are the easiest of all optimizations
because there are no campaigns to be sent - just test
pages to be rotated. That's why all our competitors do ONLY
landing page optimizations (we are the only ones to optimize
email campaigns, direct mail, and printed ads).
10. Where do you place yourself
against your competitors?
They do only landing pages, we do other media (see Question
11). They charge monthly fees and require six months
or one year contracts; we charge a one-time fee.