There can be a perception that ‘hard’ metrics are used to evaluate direct marketing, while ‘soft’ ones are used for advertising. However, as with any broad simplification, the truth is much more interesting.  Here, Darren Burnett, head of data planning at Elvis, answers some frequently asked questions about how these tools apply to direct mail.

What are hard and soft metrics?

Rather unhelpfully, specific definitions can vary, depending on who you talk to. However, to put a stake in the ground, here are some broad definitions:

Hard metrics are quantitative and measure consumers’ behaviour – the ‘who’ and the ‘what’, eg:

• Did they respond?
• Did they purchase?
• How much did they spend?
• Are they still a customer?
• How much profit did we generate for our investment?

Soft metrics tend more towards the qualitative and seek to understand consumers’ needs, motivations and perceptions – the ‘why’, eg:

• Are they aware of the brand/product?
• What do they think of it?
• What do they think of it?
• Has your communication changed their opinions?
• Why did/didn’t they respond?
• Are you on their shopping list?
• Are customers happy with their experiences?
• Would they recommend you?

What’s the relevance of soft metrics to direct mail?

The importance of evaluating the hard impacts of direct mail is well-known – its measurability is one of its strengths. We use hard metrics to track from response through sale and on to profitability to establish ROI.  Over time, they give us the impact on longer-term value, loyalty and retention.

But this still only gives part of the picture – you also need to understand the mindset behind these behaviours and the hidden impacts in order to meaningfully inform future communications. This is particularly important in understanding longer-term impacts of activity.

Shifting brand perceptions is not just the province of advertising.  Customer relationship management programmes will usually contain communications, such as welcome or marketing service communications, not necessarily designed to elicit an immediate response. 

The objective may be to increase value and retention over the longer-term, but they are also designed to change the way customers perceive the brand – ‘they do more than I thought’, ‘they go the extra mile’, etc.  The only way to find out if they worked is to ask.

In Evaluating your campaign, I covered the importance of understanding the full impact of marketing activity. You may have developed a monthly sales communication driving 2% of recipients to repeat purchase – delivering a very healthy ROI. But what about the other 98%?  Are they simply ignoring the mailings?  Or are they unhappy with the barrage of sales communications – eventually leading to a measurable hard behaviour, such as opt-out or, worse, churn. 

By the time these negative impacts are felt, it may be too late. However, research could have highlighted the issue earlier, before it started to affect profitability.

How can you track soft metrics for direct mail?

Ultimately, the approach is the same as with any research. Identify people from within the target audience and ask for their opinions of your brand and communications, and identify shifts over time.

Unlike broadcast advertising, which is focused on influencing a mass audience, direct mail is designed to influence a much smaller group, so while the principles remain the same, the available audience shrinks. However, you can identify recipients of your communication and compare to non-recipients (your no-action control /fallow cell).

Cost-effective routes to soft measurement, comparing mailed and un-mailed audiences, include:

• Introducing and leveraging customer satisfaction surveys to track key metrics, such as brand/product/service perceptions, communications perceptions, satisfaction and advocacy
• If your contacts access your products or services via the web, use online surveys
• Consider commissioning qualitative research to understand, in more depth, the impact your activity has had, to inform future strategic development

While there are many research applications within direct mail, in creative development or pre-testing for instance, the focus here is on its uses in post-campaign evaluation.

Tip It takes time to change consumers’ perceptions of your brand.  Use softer metrics to measure the impact of your programme as a whole, rather than individual communications.

Is there an alternative?

Soft and hard metrics are not irreconcilable.  Some hard metrics can be useful proxies for softer measures such as dissatisfaction.

For example, opt-outs increase can be a good indication of dissatisfaction with received communications. This can then trigger further investigation into the cause, for example, frequency of contact, poor targeting or timing, or something else.

Tip Be cautious in using easily available, free hard data as a proxy to avoid investing in research.

When monitoring hard proxies, you may be measuring a behavioural result of shifts in softer metrics, such as satisfaction or advocacy. Tipping into action can take time (even an opt-out requires a degree of effort) so, by the time it appears, it can be too late to do anything about it. 

This delay can be a significant issue when evaluating the impact of a programme that has been rolled out across the whole of your customer base.

Is it possible to link hard and soft metrics?

A key reason for tracking softer metrics is to understand behavioural drivers and to predict future actions. By identifying correlations between the two types of metrics, you can measure the financial impact of those softer shifts. This may require a significant investment in analysis.

An increasingly popular tool is the Net Promoter Score (NPS), which has managed to win the all-important battle for credibility with the ever hard-to-please financial director.

This is a topic in its own right, but to summarise: NPS was originally developed by Bain Consulting in the US, and is used to map customer advocacy to business performance. Your NPS is calculated by asking customers to state the likelihood of them recommending you on a scale of one to ten. Subtracting your percentage of detractors (least likely to recommend) from promoters (very likely to recommend) gives you your NPS.

Tip Track your NPS against business key performance indicators to determine the impact of positive and negative shifts and to benchmark performance against other brands in your sector.

Clearly, NPS is a measure affected by every contact with the customer. However, comparing the scores within contacted and un-contacted groups – at different points across the customer journey – can provide insight into how communications influence your NPS, and ultimately impact on your business.

While imperfect, NPS can be a useful tool in bridging the gap between how customers perceive your brand, and the bottom line – and help identify how communications can influence that perception.

Hard metrics are, of course, essential for understanding the behavioural impacts of your activity.  But they only provide a partial view. Soft metrics provide the rest of the picture – the insight into why people are behaving in the way they are, and how you can change it.

And in an increasingly challenging environment, such understanding is ever-more critical for delivering communications that transform the way consumers behave.
  

Need tips on evaluating the results? Find out how to successfully evaluate your campaign here