Once or twice in the past, I've talked about how your testing program needs to align with various departments in your company. For example, you need to test a design that's approved by your site design and branding teams (bright orange CTA buttons might be a big winner for you, but if your brand colour is blue, you're not going to get very far).
Or what happens if you test a design that wins but isn't approved by the IT team - they just aren't heading towards Flash animations and video clips, and they're going to start using 360-degree interactive images? The answer - you compiled and coded a very complicated dead-end.
But what about the future state of your business model? Are you trying to work out the best way to promote your best-selling product? Are you testing whether showing discounts as £s off or % off? This kind of testing assumes that pricing is important, but take a look at https://www.rolls-roycemotorcars.com/en_GB/showroom/cullinan.html which doesn't have any price information on it at all. Scary, isn't it? But apparently that's what a luxury brand looks like (and for a second example, try https://www.luxuryrestaurantguide.com/ ).
Apart from sharing the complicated and counter-intuitive navigation of the Rolls Royce site, it also shares a distinct lack of price information. Even the sorting and filtering excludes any kind of sorting by price - it's just not there.
So, if you're testing the best way of showing price information on your site while the business as a whole is moving towards a luxury status, then it's time to start rethinking your testing program and moving into line with the business.
Conversely, if you're moving your business model towards the mainstream audience in order to increase volumes, then it's time to start looking at pricing (for example) and making your site simpler, less ethereal and less vague, with content that's focused more on the actual features and benefits of the product, and less on the lifestyle. Take, for example, the luxury perfume adverts that proliferate in the run-up to Christmas. You can't convey a smell on television, or online, so instead we get these abstract adverts with people dancing on the moon; bathing in golden liquid or whatever, against a backdrop of classical music. Does it tell you the price? Does it tell you what it smells like? In some cases, does it even tell you what the product is called? Okay, it usually does, but it's a single word at the end, which they say out loud so you know how to pronounce it when you go shopping on the high street.
Compare those with, for example, toy adverts. Simple, bright, noisy, clear images of the product, repetition of the brand and product name and with the prices (recommended retail price) running constantly throughout, and at the end. Yes, there are legal requirements regarding toy adverts, but even so, no-one would ever think of a toy as a premium. Yet somehow, toys sell extremely well year after year, whether cheap or expensive, new or established brand.
So, make sure your testing is in line with business goals - not just KPIs, but the wider business strategy, branding and positioning. Don't go testing price presentation if the prices are being removed from your site; don't test colours of buttons which contravene your marketing guidelines for a classy monochrome site, and so on. Business goals are not always financial, so keep in touch with marketing!
