tl;dr
- How some things are “Not Even Wrong”.
- The importance of falsifiability.
- Making sure it exists within your business.
The Reality
I came across an interesting phrase the other day. It was the subject of a blog post by Hugo Guzman, and originally came from Wolfgang Pauli:
That’s not only not right, it’s not even wrong!
And it sounds even better in the original German: Das ist nicht nur nicht richtig, es ist nicht einmal falsch! But in any case, it’s a very important concept, particularly if you’re really committed to the kind of validated learning espoused by Lean. So how can something be “not even wrong”? Well just about everything, if you work hard enough can be structured into falsifiable tests, but that key phrase there is “work hard enough” lots of times people don’t want to take the effort to make something falsifiable, and there are certainly areas where it’s harder. Marketing and sales through social media is a great example. So you’ve got 10,ooo followers? If that’s where your analysis stops then you’re not only not right, you’re not even wrong.
So as you can probably imagine falsifiability is hugely important. I’m going to steal from Wikipedia for a bit:
By the problem of induction, no number of confirming observations can verify a universal generalization, such as All swans are white, yet it is logically possible to falsify it by observing a single black swan. Thus, the term falsifiability is sometimes synonym to testability. Some statements, such as It will be raining here in one million years, are falsifiable in principle, but not in practice.
There’s a lot of great stuff in there, but the key thing that jumped out at me was that falsifiability was a synonymous with testable. But not merely testable by a long series of confirming observations. You have to look for that one black swan. So how does this work in a business setting, particularly one involving software? Let me lay out the most common example I’ve encountered. One of the things about software development is just about every job has a set of routine tasks which comprise 90-99% of the job. And then just about every job has crises which comprise 1-10% of the job. These crises may involve debugging a particularly hairy problem; working late under high pressure; or recognizing mistakes as soon as their made, etc.
Obviously these crisis situations demand a level of ability above that required by the routine tasks that comprise the majority of a job. So how do you know if a particular employee has what it takes? Do you want to wait for a crisis to find out? And what if they do come through? Well that’s just another in the series of confirming observations. The only way to tell where the limit is. The only way to be “right” when you describe the kind of crisis an employee can handle is to keep hitting them with crisis until you find their breaking point.
So am I recommending that you subject every employee to the development version of the Bataan Death March? No. I think that you’ll probably have enough confidence in an employees crisis-handling aptitude after a few of them, but… if you have an employee who’s never had to deal with a crisis, it may not be the best thing to wait until you have an actual bone fide, extremely consequential crisis to test out things. And I’ve worked with a lot of managers who haven’t even considered what’s going to happen with the crisis hits. That’s why I think it’s a very good idea to, even if somewhat artificially, create crises early on, otherwise, when you’re evaluating how one of your employees would handle a crisis, you’re not even wrong.