Dirty KPIs, revisited

Based on the statistics provided to me by WordPress, one of the most popular entries I’ve written was Dirty KPIs.   In it, I talked about the challenges of using an activity metric like “# of miles of streets cleaned”.  Based on a workshop attendee’s assertion that one of his organization’s objectives was to “increase citizen satisfaction”, I suggested that an outcome measure such as “# complaints about street cleanliness reported every quarter” would be a better fit.  My theory was that if the organization is reaching its target for the output measure of street miles cleaned but not its target for the outcome measure of complaints, then it’s probably tracking the wrong activity and unlikely to meet its citizen satisfaction objective. 

Not long after I wrote that blog entry, a column entitled “San Jose street sweepers miss mark” appeared in a local newspaper.  In it, three separate readers complain about the local street sweepers.  If the City of San Jose had been using my suggested complaint metric, their score for street cleaning probably wouldn’t have been very high.   

Even more telling, however, is that two of the readers assert that the street cleaners don’t actually clean the streets.  This shows another flaw in using output metrics like “# of miles of streets cleaned”.  Sure, the machine passed over many miles of streets but if citizen don’t believe that the result is cleaner streets, did the organization achieve any outcome at all?  Did it have any impact? Probably not. 

This example also ties back to another popular blog entry about point of view.  Normally, we think about street sweeping from an internal process point of view.  The number of miles of streets that were cleaned represents a measure of our own activity.  However, citizens – representing the customer point of view – don’t care about how hard we worked.  They want their streets to actually be free of debris. 

No matter how efficient our processes are, if the customer isn’t satisfied with the result, it wasn’t very effective.  And we should get a poor grade on performance.

One Response to Dirty KPIs, revisited

  1. […] good example is my classic Dirty KPIs post in which I showed that an activity metric like ‘# of miles of streets cleaned’ tracks an […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: