Tim Ash to speak at eMetrics in San Francisco on May 5th, 2008

April 30, 2008

The eMetrics Online Marketing Summit is back on the West Coast. This is a no-nonsense event that has a very loyal following for good reason. I will be speaking on Monday May 5th from 5-6 pm on the multivariate testing panel.

The panel will be moderated by Google’s Jon Diorio (in charge of marketing for the Google Website Optimizer). Other panelists include David Rogers from RedEnvelope (for whom we have recently completed a landing page test), Andrew Anderson from CNET, and Matt Conahan from StubHub. It should be a solid session with lots of room for Q&A.

I will be having a book signing for my new Landing Page Optimization book at the Wiley Press booth in the exhibit hall immediately following (from 6:15 to 7pm).

Hope to see you there!


Tim Ash to write “By The Numbers” column for Search Engine Watch

April 29, 2008

I am very proud to announce that I will be a regular contributor to the “By The Numbers” expert column on Search Engine Watch (see the official press release).

As a special benefit to paid Search Engine Watch (SEW) subscribers I have included a full chapter entitled “What and How To Tune: Why Your Site Is Not Perfect” from my Landing Page Optimization Book for download in the members area. SiteTuners will also offer a complimentary landing page review to any SEW member who has a landing page with at least 100 conversions actions per day (sales, downloads, form fills).

As some of you may already know, I have been also writing regularly on the Search Engine Watch blog, and will continue to do so. The By The Numbers column offers me the opportunity to go into more depth in my articles and to also do multi-part articles. I will be focusing on landing page optimization since this is one of the best ways to make a significant impact on the economics of your online marketing program.

My columns will will alternate with those of Eric Enge from Stone Temple Consulting, a long time online marketer and industry commentator (who was kind enough to recently conduct an extended interview with me).

For those of you who are not regular readers or newsfeed subscribers of Search Engine Watch (although its free and you should be), I will cross-post some of my SEW articles and blog posts in this blog.


Taguchi Method Sucks for Landing Page Optimization

April 28, 2008

[Start of rant...]

I’m officially fed up.

If I hear the terms “Taguchi Method”, “fractional factorial”, or “Design of Experiments” in the context of landing page testing one more time, I will probably toss my cookies. I spend way too much of my time explaining to people that the Taguchi Method (at least when applied to landing page optimization) is a really bad idea. Despite this, the illusion persists that this kind of testing is somehow state-of-the-art, when in fact, nothing could be further from the truth.

Testing is composed of two important activities:

- Deciding what to test and coming up with good ideas
- Finding the best solution among your tested alternatives

People claim to get really good results with parametric fractional factorial multivariate testing approaches (of which the Taguchi Method is a subset), and they credit this to the method that they use to analyze the data. In reality, the improved conversion rates are the results of the great ideas that people include in the test. If all of your alternative landing pages designs are better then the original, it does not really matter what method you use to confirm that. Fractional factorial approaches may actually miss the best version of the landing page in your test and often lead you to a sub-optimal answer.

There is a huge mismatch between the original environment in which fractional factorial testing was developed and how it is usually applied to landing page optimization. It was basically transplanted to online marketing because it is relatively easy for a non-mathematical audience to understand, and not because of its appropriateness or fitness for the task.

The principal drawbacks of fractional factorial methods are:

  • Very small test sizes
  • Restrictive & inflexible test designs
  • Less accurate estimation of individual variable contributions
  • Drawing the wrong conclusions
  • Inability to consider context and variable interactions

Despite misinformation to the contrary, fractional factorial methods do not offer any speed advantage over full factorial data-collection approaches (such as those available in the free Google Website Optimizer tool) if you are simply planning to understand the impact of the individual variables in the test (a so-called “main effects” analysis).

If you plan on using parametric (i.e. “model building” )approaches for landing page testing you should use full factorial data collection regardless of the subsequent analysis you plan to do. It greatly simplifies your test design, and produces better estimates of the main effects.

All parametric methods (including fractional factorial) are also outclassed by newer non-parametric testing methods such as the SiteTuners TuningEngine, which can be licensed to run your own tests in-house and have the following advantages:

  • Very large test sizes (1000-10,000 times larger with the same data rate)
  • Much faster data collection (on the same data rate)
  • More accurate results (consider variable interactions)
  • Flexible test construction
  • No knowledge of statistics required

Need more info?

I have put together a 30-page extended whitepaper based on my Landing Page Optimization book and included a simple real-world case-study to illustrate the problems.

Download whitepaper“The Truth About Taguchi:
Why Fractional Factorial Multivariate Testing is Wrong for Landing Page Optimization”

whitepaper.

[End of rant...]


Search Engine Strategies NYC – Tough Crowd, High Session Scores

April 17, 2008

I speak at different conferences around the world. But SES New York is special because it is one of the largest online marketing gatherings, and also because it is known for having tough audiences. You can almost be guaranteed to have lower session scores than comparable presentations in other cities.

As I mentioned in my previous SES NYC Wrap-up post my Landing Page Testing & Tuning solos session on March 18th was very well attended (over 500 people). Of course I gave it my best shot, but I did not expect the response that I received.

My session score was a 4.75 out of 5.00 (by far my personal best)!

That means that on average the attendees thought that I was “almost excellent”.

I attribute the score to the following combination of factors:


  • Actually getting a full night’s sleep the night before
  • Excellent material from my new Landing Page Optimization book.
  • An animated presentation style (I jumped around, yelled, and even swore a couple of times)
  • Giving away $20-bills for even incorrect answers to my questions
  • A generous grading disposition from attendees still recovering from their St. Patrick’s Day festivities

Now if only there was a major holiday involving drinking the day before each of my speaking engagements, I might have figured out a way to keep my scores up …


Google Website Optimizer tool is out of beta!

April 16, 2008

Google Website Optimizer is out of beta

bye-bye beta!

Google is announcing several new developments regarding its free landing page testing tool.

Read my full post on the Search Engine Watch blog.

For additional background I am making the full 25-page GWO appendix from my Landing Page Optimization Book available as a PDF – download the Google Website Optimizer Appendix.


Overtesting & Brand Dilution

April 15, 2008

I often get questions (usually from brand/creative folks) about the effect that landing page testing may have on their precious corporate or product identity. If the visitors are in effect allowed to express their opinions, won’t that destroy the carefully constructed brand?

This question is so common, I have decided to reproduce an email exchange that I recently had on LinkedIn with Arthur Freydin (Director of Online Marketing at Madison Commercial Real Estate Services):

Arthur Freydin:

One of my biggest concerns with Landing Page Optimization is testing too much resulting in diluted brand.

Tim Ash:

What did you mean by “diluted brand”?

Arthur Freydin:

What happens when you have tested everything there is to test? When do you stop and how far do you let your visitors decide your site’s structural elements before they’ve completely stripped your brand to make their experience more streamlined and your conversions higher? I’ve read on multiple occasions about this “testing bug” and how many people will simply not know when to stop as long as they can claim that they’ve increased conversion by another .01%. How do you educate those people?

So the above will probably sound pretty harsh and you would probably expect me picketing outside your office with a “Down With LPO Sign”. On the contrary, I really believe in landing page optimization – as long as it is done in moderation. If everyone took LPO for every penny it’s worth, the web would look very similar across its verticals. I’m a huge proponent of LPO and have seen success in the past with it and intend to do it for the new company I recently started working for.

With that said, I would love to hear directly from the source (that would be you :) ) if you have encountered clients that have “overtested” their landing pages to the point of them now being unrecognizable.

Tim Ash:
You bring up several related points so I will try to untangle them and
answer from my perspective – directly from the “source” :-)

1) I agree that repeated testing of the same page is likely to diminishing improvements. The reason is that your page is getting better (harder to improve), while your ideas are getting worse (you have presumably tested the potentially powerful changes already). So at some point you should just stop and move on. There are probably other landing pages or mission-critical processes that you can apply testing to that will have a larger economic impact. You should always use that as the basis for what (if anything) to test next.

2) I don’t agree with your contention that the web will look the same across each vertical if everyone does a lot of testing. The audiences, value propositions, cost structures, business goals, and brand strengths of different companies will lead to different optimized conversion experiences for each one.

3) I think that what you mean by “brand” below is your logo system, colors, typefaces, and creative brief. It was probably designed at great expense and carefully considered to propagate the corporate identity effectively across different media and applications. If that is so, I think that it is a perfect candidate for testing. It should not be a sacred cow. If your visitors prefer an ugly Frankenstein Monster to your beautiful creation, then your creation should be thrown in the trash. If the brand was so important, it would contribute to the experience and remain part of the winning page. It comes down to expectations. A better converting page by definition meets the expectations of
your visitors better. The rest is ego investment in your own creative work and should be jettisoned. Period – end of story.


Landing Page Optimization Book Hits #3 on Amazon’s E-commerce Bestseller List

April 8, 2008

When I wrote my book I had no idea what to expect. Did anyone else care about landing page testing? Would anyone buy a book on the subject?

I must say I have been extremely flattered and gratified by the response to the book. It has gotten terrific feedback from readers.

The book is currently #3 on the Amazon e-commerce bestseller list and is in the company of books by two online marketing gurus whom I have a lot of respect for (Seth Godin and Avinash Kaushik)!

The rank changes hourly, so I have become of bit of a “day trader” and like to watch the pot boil.

See current Amazon e-commerce bestsellers.


Marketing Sherpa Landing Page Handbook – 2nd Edition

April 4, 2008

A five hundred dollar book and it’s spiral bound – you’ve got to be kidding!

The good news is that it’s only $497 and it’s worth every penny. The new 2nd Edition of Marketing Sherpa’s Landing Page Handbook is finally out. For those who have read the 1st edition, this will be a long-awaited update. The field of Landing Page Optimization has grown and changed dramatically, and this new edition keeps up with the changes. It is much more in-depth and contains very detailed information about industry statistics as well as over 100 practical case studies that will give you many ideas for your own landing page tests.

My own Landing Page Optimization book was just recently released by John Wiley & Sons Press and is the first one to offer a comprehensive guide to landing page testing. The Marketing Sherpa handbook is a perfect companion book. But don’t take my word for it – the Handbook comes wth a 100% satisfaction guarantee. Highly recommended.


HackerSafe and The Power of Trust

April 4, 2008

I am often asked by people what sorts of items I would recommend testing on a landing page. There are no universal truths and your mileage may vary. However, there are common themes that work well across a range of industries. One of these is the use of trust and credibility indicators. These reduce anxiety for your visitors and increase their confidence when interacting with your landing page.

A common type trust and credibility indicator are various safe shopping seals. The leader in this area is McAfee’s HackerSafe trustmark. In most cases, adding the trustmark to your website will increase conversion rates by several percentage points. But just adding the trustmark to your page is not enough. You also need to consider which version of the trustmark to use (they come in several shapes and colors), and where on your page it will appear. Many companies mistakenly put their trustmark “below the fold” on their page. By doing this they make the trustmark invisible and practically useless. We always suggest putting the trustmark prominently near the top of the page.

But even this is not enough. The specific location within the page header can result in additional improvements. In a recent test we tested different positions and trustmark versions on the LuggagePoint website and were able to find an 11% increase in revenue per visitor and a 5% increase in conversion rate simply by changing the position of the trustmark in the header.

Read the LuggagePoint case study here.

As a McAfee HackerSafe partner SiteTuners.com will run a free multivariate test on any site with certain minimum traffic levels to determine which placement on the page (out of 4 possible positions) and HackerSafe trustmark image (out of 6 possible versions) results in the best performance.


Search Engine Strategies New York 2008 Wrap-up

April 2, 2008


Search Engine Strategies flew by in a New York minute. Media interviews, business meetings, terrific show content, and late night partying kept me up until the wee hours of the night.


View my SES NYC 2008 Photo Album.

Highlights of the show included a keynote by Nick Carr about his new and thought-provoking book “The Big Switch” (see my blog entry on Search Engine Watch).

My session on Landing Page Testing & Tuning was very well attended and received (over 500 people in the room).

A subsequent book signing for my new Landing Page Optimization book in the exhibit hall sold out of book copies in about a half hour. This seems to be the year that Landing Page Optimization is really catching fire.

In a rare public display I was caught by Greg Jarboe on video doing salsa dancing as he interviewed me about the book.