Darren Burnett, head of data planning at Elvis, outlines the key questions you need to answer to carry out an effective campaign evaluation.

Evaluating a marketing campaign is as vital as planning it. After all, unless you know how effective it has been, how can you improve or build on it next time? In the second of two articles, we explain how to evaluate your marketing campaign.

Can you spread the word early?

• Don’t get over-excited by initial results – good or bad. You planned when you could meaningfully deliver the evaluation for a good reason – so those results could change dramatically.

Tip Ensure that the planned delivery dates are well-publicised in advance – remember, it is important to wait until you have the full picture. This will help manage expectations.  

What do the numbers say?

• Use statistical significance to ensure that the results are meaningful, but be clear about how confident you need to be in the result.
• Balance the need to obtain actionable learning against the level of risk implicit in a wrong decision.

Tip If a result falls only a little short of your required confidence level, remember that this doesn’t necessarily mean that the tested element failed – only that you cannot be confident in the witnessed outcome. It may be worth running the test again to gain additional volume… but bear in mind that this will probably mean delaying another planned test.

Why did you get this result?

• What happened is only part of the story, but we also need to understand why.
• It's time to look at those results in context and understand what impacted on this result. Competitors?  Market conditions? Operational issues?
• This is not about trying to find excuses for a poor performing campaign – it is important to be as thorough on successes as failures – it is about getting to the heart of what has happened.

Example Knowing that incentive A beat incentive B tells us only that we should roll out incentive A. Understanding that this incentive aligned better with your product and the target audience than what was a more generic voucher offer helps us to better select new incentives in the future.

Do we need to find out more?

• Test your findings against your common sense – if something really jars, there’s probably a reason for it. Dig deeper.
• Use further analysis to disaggregate the results – for example, can we profile responders to see if certain audiences were more responsive than others?
• Consider research to better understand the drivers behind this result.

Example You have developed a third party offer that aligns well with your target audience and has researched well. However, in live testing, it bombs.  Before we bin the idea, we need to understand why it appears to have failed. Was there any problem with handling response and fulfilment? Were the benefits clearly conveyed in the communication – or could it have been missed completely, for example, it was a secondary piece within the pack? Or have we drawn the wrong conclusions from that original research? 

What are we going to do next?

This is the most important section of any evaluation
• All that work is almost useless if it doesn’t help inform strategy going forward.
• It's critical to be absolutely clear about how these findings will impact on future activity. 

Tip Don’t just think about what this means for the next communication. Include sections on what new elements should be added to the test plan and what analysis or research is required.

Delivering meaningful evaluation is critical to ongoing marketing success, and it merits attention. Remember:

• Great evaluation comes from great planning. Do the work upfront and it will save an awful lot of time and effort later. Without it, you may be unable to get the full picture at all.
• Curiosity and common sense are key evaluation tools. Make sure you look beyond the numbers to understand why you got that result. And if the result is counter-intuitive, keep looking until you’re confident the findings are genuine.
• Evaluation is about looking forward, not backward. Always ask, what will we do differently next time?

Need tips on which tools to use for evaluation?

Find out more about using soft and hard metrics here

Find out more about using test and control cells here