Wednesday 17 August 2011

KPIs, smoke and mirrors

Captains of industry and latter-day politicians get very horny over KPIs or to give them their full unabbreviated title Key Performance Indicators. Unfortunately as with most management tools they are a sham. They don’t measure performance per se, rather they measure people’s ability to ‘play the game’ and to massage and manipulate figures. Those that are ‘clever’ find ways to influence outcomes. Those that either don’t care or don’t realise that it is a game rarely do so well in the measurement stakes. Sadly important decisions are made on the back of all this false data. People’s livelihoods are affected by them. It certainly happens to a degree in the company I work for and I suspect it happens in varying degrees up and down the land.
If targets are set and people don’t meet those targets it doesn’t necessarily mean they have ‘failed’ even though it might be seen as failure. It could be that:
  • people are too busy doing a really good job that they don’t have time to facilitate the generation of correct data
  • the overall data collection process is flawed
  • someone else is entering data incorrectly
  • data is being misinterpreted
  • the metrics used are not well defined and/or have not been communicated to those concerned
  • data of different standards and from different sources is being accepted as ‘like for like’
  • people concerned aren’t doing their job properly
  • any permutation of various of the above
  • Other factors that I’ve failed to consider
One of the biggest problems with data collection is inconsistency. In large organisations data will invariably arrive from a number of different sources. Different branches of the organisation will interpret what information is required each in their own little way. Even if information is gathered from electronic systems it is subject to contamination; unless you have got very basic black and white true/false type data it will be flawed. The data entry if made by humans will vary. The more sophisticated the data is the more chance it has of being incorrect. These measurements will be further corrupted where you have government departments that collect data from outside organisations. The variables of interpretation offer so much scope for erroneous answers as to render many findings pointless. KPIs can only ever be accepted as giving an approximate overview or flavour at best. They can only ever be considered as a licking your finger and holding it up to the wind type of approach. 

In my experience rarely do KPIs tell it like it really is and far too much weight is given to them when making important decisions. Essentially KPIs are just statistics and we know exactly what that means. There really is no replacement for quality, hands on, management; people who know the job and being trusted with getting it right.
One of the biggest problems with data collection is inconsistency. In large organisations data will invariably arrive from a number of different sources. Different branches of the organisation will interpret what information is required each in their own little way. Even if information is gathered from electronic systems it is subject to contamination; unless you have got very basic black and white true/false type data it will be flawed. The data entry if made by humans will vary. The more sophisticated the data is the more chance it has of being incorrect. These measurements will be further corrupted where you have government departments that collect data from outside organisations. The variables of interpretation offer so much scope for erroneous answers as to render many findings pointless. KPIs can only ever be accepted as giving an approximate overview or flavour at best. They can only ever be considered as a licking your finger and holding it up to the wind type of approach.
In my experience rarely do KPIs tell it like it really is and far too much weight is given to them when making important decisions. Essentially KPIs are just statistics and we know exactly what that means. There really is no replacement for quality, hands on, management; people who know the job and being trusted with getting it right.

No comments:

Post a Comment