Category Archives: Measurement

Keeping Communication Measurement On-Budget

photo cc-by it comes to marketing and communications, information has a value that is very straight forward to calculate. All too often the cost to retrieve and share that information exceed its value, leading to a general consensus that measurement is expensive; a luxury that may fit in someone else’s budget but, with the limited resources in your hands, the money is better spent doing something rather than observing something.

Measurement shouldn’t be breaking the bank. If it is, then you are making one or more of the following mistakes.

You’re measuring and reporting too frequently

Those of you who are continually refreshing your social news feeds and compulsively checking your emails will understand what I’m talking about regarding the fear of missing out. There is an underlying anxiety that builds when we don’t know something and the need to check and check and check can be overwhelming. All the worse when this desire for an update is coming from several rungs up the ladder. This continual call for a report is being done to satisfy a curiosity rather than accomplish anything and it churns quickly through the dollars.

Your data retrieval ought to occur at an interval in which meaningful change will have occured. Rather than asking, are we there yet? “No.” are we there yet? “No.” are we there yet? “No.” are we there yet? “No.”are we there yet? “No.” are we there yet?

…it would be better to ask at the start, how long do we expect this trip to take? What’s the next turn? What’s the next stop? At each turn of the road a quick exam as to the conditions ahead. At each stop a quick view as to whether you’re making the time you thought you would and is there a change in route necessary? A few checks when they’re actually needed instead of an exhausting litany that can’t help but become just a droning background annoyance.

While your cadence for measuring should match the rate of any significant change occurring, your reporting should come in advance of any decision making process. That is when the information is actually needed. It makes no sense for hourly reporting to be going on through the wee small hours of the morning if everyone who could reasonably do something with that information is fast asleep. Better a single report timed to arrive as the decision makers are starting their day. Better yet, timed to arrive a few hours in advance of the weekly meeting where they make the actual decisions.

You’re being too precise in your measures

Do not forget why you are taking these measures. You are trying to make better decisions and this is the information that will help you do so. A lot of people end up chasing the numbers and losing sight of the decision.

Let’s say you are testing messaging in a new market in advance of a large campaign, and a quick poll of a few dozen people shows that 90% find your phrasing offensive. Well, a quick poll’s not very precise, is it? So let’s do some more formal focus group testing of a couple hundred people. From that testing, 85% found it offensive. That number’s different from the poll, and how precise are focus groups anyways? This campaign is important, and you really want to be sure, so you splurge on a large, random telephone survey of over 10,000. Now you have a really precise number. You know with 95% confidence and a small margin of error that the number who find your messaging offensive is 72.43%.

What was the acceptable number? Maybe you’re an edgy brand that embraces controversy and you’re willing to accept a number of people being turned off in exchange for the exposure. Most brands entering a new market want to be putting their very best face forward and would have little to no tolerance for offending potential customers.

Before the random survey, before the focus groups, before all of that additional expenditure chasing after a more precise number, it was obvious that there was a problem. The difference between 95% and 85% and 72.43% were not going to change that.

You are collecting information in order to make a decision. Once the decision is clear, you have all the information that you require.

You’re not measuring against your objectives

What are you trying to achieve? What is the change in the market that your actions are intended to make? What decisions need to be made? If you don’t know the answer to these, you will not know what you need to measure.

Not knowing what to measure leads people to try and measure everything in hopes that they will capture something of value. The resulting reports are number soups filled with some information that’s interesting and a great deal that is irrelevant. But without knowing your objectives, there is no way to know what information is important.

It may be interesting to know that 5% of your website visitors are using their mobile device from the bathroom: an odd factoid. However, if you’re in the middle of a campaign and you know that 30% of your visits are coming from a single media market where you’ve been doing a heavy push on daytime radio, that’s important. That will help you make decisions as to what to do next and where to push your resources.

Much as there’s always more precision to be had, there will always be more information to be had. You could keep at it until you’ve burned through the budget ten-fold and still have new avenues to chase down for more information. So to keep from breaking the bank, you want to focus your measurement on the important over the interesting.

In short…

  1. Start from a solid objective and ensure your measures are against that objective.
  2. Don’t let perfect be the enemy of good. Choose your tools and methods so that you get enough information to make your decisions accurately.
  3. Set a cadence in lockstep with the precision you need and report on it as the decisions need to be made.

Follow those three steps and you’ll not only have the information you need to make better decisions, but the extra dollars in the budget to act on that information.

cross-posted from Linked-In Pulse

Are You Sure Your KPI is a KPI?

Invasion of the fraudster metrics

Most of us have been in a meeting of this sort. You’re working out the details of a program and the conversation turns towards metrics and the need for KPIs. By the end of the meeting you have a list, somewhere between thirteen and twenty-seven different KPIs. Someone is tasked with putting together an excel sheet to track everything and you commence an ongoing saga of trying to visualize the results onto less than three slides in a PowerPoint deck.

Drop the PowerPoint and step away from the Excel file.

I don’t need to know your brand, or your tactic or the planned executions.; but I can tell you right now that you’re doing it wrong.

If you’ll let me help you, it’s time to stop the KPI insanity.

All KPIs are metrics, but not all metrics are KPIs

KPI - you keep using that word.  I don't think it means what you think it means.

Or put another way; not all things that can be counted, count. KPI is an acronym for Key Performance Indicator. It is the one measure that shows that you have achieved what you set out to achieve.

It is natural for people to want to include more. When you see that jam-packed PowerPoint slide filled with charts and graphs and tables filled with numbers, it feels all very impressive.

Look at all the numbers! Everyone must have worked ever so hard for there to be this many numbers and charts moving up and to the right.

But it’s all just surface razzle dazzle. Those that know what they’re doing are going to be less than impressed by attempts to baffle your way through the numbers. And those people tend to be C-suite execs with their fingers on the purse strings for next quarter’s budget.

You can’t set your KPIs before you set your objectives

Putting the cart before the horse

Most people don’t realize this – as everyone tends to skip past terms of service and just check off ‘OK’ – but it is a basic part of every single employment agreement that anyone who recommends setting KPIs before objectives are identified must go up to the rooftop on a rainy day and run forty laps around the circumference of the building.

Okay. So, maybe not. But it would certainly stop short the useless exercise of defining what success looks like in advance of defining what you’re trying to do.

A KPI is a metric that the program hinges on; a metric inexorably linked to your objectives. When you look at your KPI it is a no-brainer as to whether you achieved what you set out to do.

 Your KPI is a unit of measure, not a specific measure

When asked, “What are some good marketing KPIs if the objective is to create a positive association with our brand?” you will never find that the answer is 27. Or 1.32. Or 1,589.

Your KPI will always be a unit of measure, and not a specific number. The specific number is your target or goal.

For example, if you ran a shop, your KPIs might be cash flow and net profit. These are the indicators that your business is thriving or not. Within cash flow you would set a specific number that ensures you can maintain operations as a target. If you have an eye on expansion you would set a goal within net profit that will allow you to achieve that expansion.

You should be able to count your KPIs on one hand

If you need more than one hand to count your program’s KPIs, then you have let a metric slip in masquerading as a KPI. You need to be ruthless with respect to KPIs and slice out measures that people toss in because, “wouldn’t it be interesting to know?

Your KPIs should all fall into the category of, “we absolutely have to know.” Rare is the program that needs more than one measure to determine pass or fail. I’ve yet to encounter a program that truly required more than a half-dozen measures to track performance.

The easiest way for you to suss out the frauds is to honestly ask yourself and your team, “If this KPI came in really high, or came in ridiculously low, would it change our reaching the objective?”  If it doesn’t matter, then it shouldn’t be counted.

Why does it even matter?

Why get picky about what’s a metric and what’s a KPI and if they ought to be numbers or not? What’s all the hub-bub?

Information is vital. If your information is poor, then your decisions will likewise be poor. If you end up, like all too many of us do, tossing aside all of the information because it’s just a hodge-podge of number soup you end up acting purely on instinct and gut-feelings.

Steering the ship based on your gut may keep you free and clear of the rocks most of the time, but the bigger your ship and the more people you carry along with you, the more tragic a single misstep can become.

As vital as information is, however, every dollar spent on watching what you’ve done is a dollar you can’t spend doing something.

Instead of measuring everything, the smart business person looks to what they are trying to achieve and focuses their resources towards measuring that and measuring it well.