Testing Hopes for 2014


Christmas is a ”Lichtfest” for us in the North. Daytime, at this time of year, only lasts a few hours and the sun never really rise on the horizon. Christmas reminds us that light days will return and it’s time to look ahead on the year to come.

I have two hopes for software testing for 2014:

  1. I hope we will stop looking for simple explanaitions why something failed: The product, the testing, the development.
  2. We cannot expect all managers to be testing experts, so we need better documented and qualified testing practices (in various contexts) in order to support better top management software testing decisions.

Looking back on 2013…

I had a busy 2013, privately as well as professionally. Let’s Test in May was fantastic! A few weeks later, I gave a successful lecture on Context Driven Testing in IDA-IT.

I have for long wanted to link my favorite philosopher Niels Bohr to testing. Denmark celebrated the 100 year anniversary of Niels Bohrs articles on the atom model this year. Niels Bohr was a Nobel Prize wining physicist, but more than anything, he was a philosopher – my favorite philosopher by far.

My second favorite is Nassim Taleb. Taleb published his new book Antifragile in late 2012, and I read it this year. But it was his previous book Black Swan that made me a fan.

In chapter 12 of Black Swan, Taleb criticizes historicism: Always wanting to find causes of why things happen. That happens a lot in testing too:

  • ”Why was that bug in the system!?”
  • ”Why didn’t test find it!?”
  • ”Who blundered!?”

Taleb points out that explaining an event is just as difficult as predicting the future. He argues that any logical deductions and computations involved in analysing an event will yield random results.

Good managers knows that appreciating and handling a team’s frustration over something not going as planned is important, but we are too often committing the error of turning a psychological healing process into a development system, mindlessly making up apparently deterministic explanations for the unexpected – the random.

Randomness and historicism

Randomness is actually two different things: (1) Indeterministic mathematical radomness. (2) Something that is acting chaotically, but still according to deterministic laws.

The ”butterfly in india” is an example of a chaotic, but deterministic chain of events: It is said that the beating of a butterfly’s wings in Delhi can cause a thunderstorm in North Carolina.

According to Newtonian and relativistic physics, determinism is a fundamental property of nature, but since most of the events involved in the forming of the thunderstorm are outside our reach, we won’t be able to reconstruct the event completely anyway.

This is perhaps where Taleb and Bohr might disagree, since Bohr did not believe in determinism as a fundamental property of nature.

With quantum physics, Bohr, Heisenberg, Pauli and other pioneers were able to show that events on the nuclear level do not follow rules of causality. An electron, for example, moves from one energy level to a lower, releasing a photon, spontaneously.

”So what? We’re not living in microcosomos. Butterflies don’t move electrons, they set complete molecules in motion. Causality should still apply on any observable level.”

This is a valid counter argument, but Bohr, in several of his philosophical writngs, points out that the lack of casualty on the subatomic level does in fact affect the macroscopic level: There are many amplification systems in nature, which amplify single quanta of energy into macroscopic effects. One such is the human eye, which can detect single photons and amplify it as a stream of information sent to the brain, where it can trigger actions. Obviously there are lots of such amplification systems in the brain and our bodies, so maybe there’s no such thing as determinism in people? And in nature in general, for that matter.

Does having a bad childhood make someone bad?

Se we’re essentially left with a world of repeatable patterns. Statisticians know that children of poor parents will usually be poor themselves. That is a well known pattern, but does it work the other  way too? Does a bad childhood make you a bad person?

Obviously no. The pattern cannot be linked to the individual, per se.

But that doesn’t mean patterns aren’t useful: Patterns simplify reality, and simplification is necessary in all planning and management.

Many projects have contracts which are negotiated several years before the testers start. Such contracts often specify which kinds of testing should take place e.g. how acceptance testing should be carried out.

Now, we can’t expect all IT contract managers to be testing experts, but if we can document research evidence of the usefulness of e.g. exploratory testing, we’re much more likely to be able to convince them to use it constructively, even when they’re working on the early planning phases.

Happy 2014!