Let’s be honest we all love measuring stuff. You don’t need to be a mathematician or an engineer to see the benefit of putting a simple number on complex things.
But metrics - no matter how cleverly designed - can be misleading. In addition to that, there will always be a small portion of people who will try to game the metrics to their advantage.
Let's have a look at a famous example from history.
The soviet union needed nails for its industry. Nothing easier than that, said one comrade and installed a straightforward metric that would reward each factory at the end of the year by the number of nails produced.
More nails - more rewards. Unfortunately, this is where KISS (keep it simple stupid) horribly failed. What happened was that more than one factory almost immediately specialized in producing extremely tiny nails. After all, you can produce more nails with the same amount of steel and workers if you just reduce the size.
But of course, the soviet union didn't need those tiny nails!
Don't you worry, said our comrade, I got you covered. Refreshed with his new learnings about metrics he understood his mistake and corrected it.
Now each factory would be rewarded by the total weight of all Nails produced - no further need for tiny ones.
Good old comrade solving one problem with another… Guess what happened.
Now even more factories gamed the metric by producing extremely huge nails. After all, if you just focus on weight - one big nail will get you further than a handful of normal-sized ones and will be produced way faster.
I guess that's when our friend and comrade gave up and the central party just told each factory exactly what to do. Good old communism at its best.
Godharts’s law states:
“When a measure becomes a target, it ceases to be a good measure"
You can (unwillingly) incentivize behavior that doesn’t benefit the company or in the worst case actually causes harm.
In addition to that, you might create a culture that will be very hard to correct further down the path. Famous stories have been written about the toxic culture of the early days of Uber and how long it took the new management to correct it.
Metrics can be misleading - what you measure is what you value.
Next time you wish for or even demand a metric from your team - be careful and maybe sleep a night on it. If you still believe in numbers and measurements then it might be better to have multiple metrics instead of just one and here is why:
They are all going to be “wrong” but in different ways, so you can learn from them, keep what is working and correct the rest.
Multiple and complex metrics are way harder to game so people will (hopefully) spend less time gaming the metrics and more time achieving actual goals.
They empower people to go in their preferred direction since multiple things will be rewarded instead of just one big number that needs to be achieved.
Note to the reader: If you are reading this for facts and learning then you can stop here and I wish you a good day. If in addition to learning entertainment is what you seek - carry on.
Interestingly humans are not the only creature that tries to game the metrics for their advantage. One day at SeaWorld the trainers had a great idea.
Why not use the dolphins themselves to clean the pool from the litter? All they needed to do was to reward them each time they picked up a piece and brought it to the trainers. Since dolphins are extremely smart it didn't take them long to get the gist. The pool was cleaner than ever and the dolphins were happy with all that extra “reward-fish” in their bellies. That is for a while at least. Because then things started to happen. One of the Dolphins thought - since they were being rewarded in a piece by piece metrics - why not tear the litter apart into smaller pieces and get more fish out of it? Wow. Pretty clever right? But wait it gets better worse.
Sadly sometimes seagulls crash into something and fall into the water and drown. Well since the trainer also accepted dead seagulls as litter and rewarded the dolphins for it with fish - the connection was made: dead seagulls equal tasty fish.
It didn't take the dolphins long before they were stockpiling fish to attract seagulls and deliberately kill them. Well, that escalated pretty quickly!
Once the trainers saw what was going on it was very hard to correct the behavior. It looked like the dolphins found a new hobby to kill their time (and their feathered friends).
This post was inspired and is based on episode 240 of the “Rationally Speaking” podcast from Julia Galef and her guest David Mannheim.
Have a successful day
GNF
Further reading: Check out Peter Singers’ book: Bureaucracy: What Government Agencies Do And Why They Do It
PS. If you’re mentally hunted by the dolphins luring in seagulls …welcome to the club. I’m thinking about starting a self-help group …