Darren Burnett, head of data planning at Elvis (Campaign and Marketing's Direct Agency of the Year 2008), outlines the key questions you need to address to successfully measure a campaign.

Measuring a marketing campaign is as vital as planning it. After all, unless you know how effective it has been, how can you improve or build on it next time? In the first of two articles examining the consequences of a marketing drive, we explain how to measure its success.

What are you trying to achieve?

• Clearly define learning objectives and clarify anywhere that these conflict with the broader campaign objectives.

Tip Be completely clear about what you will do differently as a result of the testing. Testing for testing’s sake is a fool’s errand – and a waste of marketing budget.

How can you measure performance?

• Define metrics and put the tools in place to monitor them.
• Commission any necessary research, such as pre, during, and post-campaign tracking to assess softer measures, such as brand awareness or consideration.
• Determine the mechanic(s) for attributing behaviour to the campaign
  Trackable responses (click-throughs, phone numbers, etc) are highly attributable but relying only on these discounts responses to untrackable channels, such as the main website.
• Matchback provides a fuller picture, but requires a robust control strategy to remove unprompted ‘responses’ and can only be used in direct communications.
• Go beyond simple response and cost per response to understand profitability, wherever possible. Ordinarily, the primary metric on any activity should be the return on marketing investment (ROMI) or ROI – how much profit has been generated for each marketing pound spent?

Tip Review the way in which these metrics will be reported now – it will be much easier to influence reporting at this stage than later, while everyone is clamouring to know how well your activity has performed!

Are you looking at the big picture?

• If this activity is part of a wider through-the-line campaign, how are the other elements being evaluated – and how will these be brought together?
• Consider the broader impacts of any activity and factor this into tracking.

Tip Monitor metrics such as opt-out rates, satisfaction/advocacy scores, retention rates within contacted and uncontacted groups to understand the wider implications of your activity.

Example Frequent, standardised hard-sell communications sent to your customer base may well deliver the best immediate return on investment. However, unless a wider set of metrics are included, you might miss the fact that this is also increasing opt-outs and churn within your customer base, and actually having a negative impact on your bottom line!

How can you be sure what you’re measuring?

• Isolate the elements being tested. For example, if you're testing a new creative, ensure that all other elements are standardised – target audience, timing, offers, incentives and so on.
• Also, include a non-contact cell to use as a campaign control. It is essential to be able to understand – and strip out – what level of behaviour, for example sales, you would have received without that investment. This is particularly critical with very warm audiences, such as customers who are likely to repurchase without prompting – and when in a noisy environment such as a multi-media campaign, where there are many other influences on your target audience.
• It's essential to ensure your control cells are sized to deliver a statistically robust result. This is not as simple as arbitrarily assigning x% of your available volume. The volume required will depend on predicted response rates and how confident you need to be in the result. Speak to your analytical support or your agency about how to calculate these volumes.

Note It's critical that cells exactly reflect those they are being compared to except for the element being measured. Any differences will need to be accounted for in the final evaluation – making evaluation highly subjective.

Tip Build your test plan as a tree, breaking out each individual component of the test, for example, targeting, incentive, creative. This structure will help identify areas where more than one component is being tested at once (as well as helping show where the structure can be simplified to remove repeated tests).

Example A champion creative delivers 4% response when mailed in January. A new creative is tested in July, delivering 3%, indicating it to be less effective than the champion. Except that building in a control cell of the champion, tested at the same time, would show it to only deliver 1.5% (because of market changes). The test is, in fact, a big success and without the control, the wrong decision would have been made.

How soon can you make a call?

• The point at which a campaign matures (that is, when all impact has been delivered) varies significantly, especially by product and media. For example, email tends to deliver a response very swiftly, while a direct mail piece can still be delivering response many weeks after delivery. And while a seasonal travel offer might deliver all associated sales within a couple of weeks, a mortgage or automotive communication might still be having an impact after three months.
• So, you need to understand how response curves look for your market and medium and plan accordingly. Schedule that evaluation on the timing plan and communicate it to the team to manage expectations and ensure it isn’t forgotten until the last minute!

Tip If response curves are stable, it may be possible to evaluate early and forecast the mature result accurately.

Example If 80% of responses for your direct mail are ordinarily received within two weeks, with the remaining 20% trickling in across the next four, then we can speed up evaluation. After two weeks, we can project that the 2% response so far received will ultimately mature to 2.5% (that is, 2%/80%). Of course, this then needs to be reviewed at six weeks to ensure that forecasts are accurate and stable

Who’s doing what?

• Designate an evaluation champion – the ultimate owner of the final evaluation document(s).
• Agree who will be responsible for extracting the numbers from the database.

Tip Invite a broader team to feed in and provide context in terms of other comms, competitor activity, market conditions – once you've done that, you're ready to go out there and measure your feedback.

And once you've measured your success, it's time to look at the results. Find out how to successfully evaluate your campaign here