Strategy - Short term testing vs long term results

Are you as addicted as I am to a/b testing? If so, you need to remind yourself from time to time to step back and look at longer term results.

The reason for this is that an a/b test will always show the "winner" of a short term test, but does not reflect what you're doing for your business in terms of customer loyalty or lifetime customer value (LCV).

For instance, I guarantee you that an aggressive sales email with a discounted offer that expires quickly will always outperform a less aggressive offer. With that in mind, however, an overly aggressive offer will hurt your brand and reduce the trust your best customers have in your offering.

So, how to protect against this? First off, use some common sense! If your home page or emails are starting to look like a multi-level marketing pitch, you've gone too far. Secondly, step back from looking simply at revenues and have a look at other critical indicators, such as the abandonment rate of various pages and the unsubscribe rate from your newsletters.

If your company has the data (which it should if you're serious about staying in business), step back and have a look at average annual sales per customer, as well as month over month, quarter over quarter, or year over year trends.

Remember, just because a single test seems to validate an overly aggressive tactic, that doesn't mean it's the right thing to do.