With Cynefin, I can justify skepticism about inappropriate approaches and create better ones

As testers we need to better understand and be explicit about problems in testing that don’t have known, clear, or obvious solutions. Cynefin can help by transforming the way we, our teams, and our stakeholders think about testing problems.

Ben Kelly and James Christie has written very good blogs about Cynefin and testing. Liz Keogh was one of the first to write about Cynefin in development. At the bottom of this post, I have included a video with David Snowden and a link to an article I found interesting when I read it.

With this blog post I’m sharing elements of my own understanding of Cynefin and why I think it’s important. I think of Cynefin itself as a conceptual framework useful for comprehending dynamic and complex systems, but it is also a multi faceted “tool” which can help create context dependent conceptual frameworks, both tacit and explicit, so that we can better solve problems.

But before diving into that (and in particular explain what a conceptual framework is), I’d like to share something about my background.

Product design and the historic mistakes of software development

I used to study product design in university in the early 90’s. Creating new and innovative products does not follow obvious processes. Most engineering classes taught us methods and tools, but product design classes were different.

We were taught to get into the field, study real users in their real contexts, develop understandings of their problems, come up with prototypes and models of product ideas, and then try out these prototypes with the users.

Discussing an early draft of this post with James Christie, he mentioned that one of the historic mistakes of software development has been the assumption that it is a manufacturing process, whereas in reality it is far more like research and development. He finds it odd that we called it development, while at the same time refusing to believe that it really was a development activity.

SAFe, “the new black” in software delivery, is a good example of how even new methodologies in our industry are still based on paradigms rooted in knowledge about organizing manufacturing. “The Phoenix Project”, a popular novel about DevOps states on the back cover that managing IT is similar to factory management.

What I was taught back in the 90’s still help me when I try to understand why many problems remain unsolved despite hard work and many attempts on the opposite. I find that sometimes the wrong types of solutions are applied, solutions which don’t take into consideration the true nature of the issues we are trying to get rid of, or the innovations we’re trying to make.

Knight Capital Group, a testing failure

The case of Knight Capital Group is interesting from both innovation, risk and software testing perspectives, and I think it exemplifies the types of problems we get when we miss the complexity of our contexts.

Knight Capital Group was one of the more aggressive investment companies in Wall Street. In 2012 they developed a new trading algorithm. The algorithm was tested using a simulation engine, I assume to ensure to that stakeholders that the new algorithm would generate great revenues.

The testing of the algorithm was not enough to ensure revenues, however. In fact, the outcome of deploying to algorithm to production was great losses and the eventual bankruptcy of the company after only 45 minues of trading. What went wrong?

There are always several complementary perspectives. SEC, Securities and Exchange Commission of the U.S.A.:

[…] Knight did not have a system of risk management controls and supervisory procedures reasonably designed to manage the financial, regulatory, and other risks of market access […] Knight’s failures resulted in it accumulating an unintended multi-billion dollar portfolio of securities in approximately forty-five minutes on August 1 and, ultimately, Knight lost more than $460 million […]

From a testing perspective, it’s interesting that the technical root cause of the accident was that a component designed to be used to test the algorithm by generating artificial data was by some kind of mistake deployed into production along with the algorithm itself. This test component created a stream of random data and the effect was that the algorithm issued purchase orders for worthless stock.

It is paradoxical that the technical component that caused the accident was designed for testing, but it is not uncommon that software testing is often focused on relatively obvious, functional and isolated performance perspectives of the system under test.

Cynefin transforms thinking

Let’s imagine you’re the test manager at Knight and you choose to use Cynefin to help you develop the testing strategy for the new algorithm. David Snowden talks about Cynefin as a ‘sensemaking tool’ and if you engage Knights’ management, financial, IT-operations, and development people in a facilitated session with a focus on risks and testing, I’m pretty sure the outcome would be the identification of the type of risk that ended up causing the bankruptcy of the company, and either prevented it by explicitly testing the deployment process, or made sure operations and finace put the necessary “risk management controls and supervisory procedures” in place.

I think so because even with my limited experience so far, I have seen how Cynefin sessions are great for forming strategies to deal with the problems, issues, challenges, opportunities etc that a team is facing. It helps people talking seriously about the nature of problems, transforming them, and to escalate things that require escalation.

Cynefin seems to be efficient breaking the traditional domination of linear and causal thinking that prevent problem solving of anything but the simplest problems.

My interpretation of what is happening is that Cynefin helps extend the language of those participating in sessions, and in the following I’ll dive a bit more into why I interpret it that way.

Language and Conceptual Frameworks

Language is an every-day thing that we don’t think about, yet it is the very framework which contains our thinking. While we can know things we cannot express (tacit knowledge), we cannot actively think outside the frames our language creates.

Many philosophers have thought about this, but I’d like to refer to physicist Niels Bohr (1885-1962) who in several of his lectures, articles, and personal letters talks about the importance of language. Poetically, and paraphrasing him from my memory, he describes language as the string that suspends our knowledge above a void of endless amounts of experiences.

In a particular lecture “The Unity of Science” given at Columbia University, New York in 1954, Bohr introduce language as a “conceptual framework” and describes how Quantum physics is an extension of the previous conceptual framework used in physics:

“[it] is important […] to realize that all knowledge is originally represented within a conceptual framework adapted to account for previous experience, and that any such frame may prove too narrow to comprehend new experiences.”

And:

“When speaking of a conceptual framework, we merely refer to an unambiguous logical representation of relations between experience.”

Quantum physics is more than new laws about nature. Rather, it introduced new and complimentary concepts like uncertainty, and non-deterministic relations between events. The extension was made for quite practical purposes, namely the comprehension of observations, but has turned out to be quite useful:

“By means of the quantum mechanical formalism, a detailed account of an immense amount of experimental evidence regarding the physical and chemical properties of matter has been achieved.”

The rest is history, so to speak.

Why is this relevant to software testing and the talk about Cynefin? First of all, I think that the conceptual frameworks based on the thinking developed during industrialism are far from capable of explaining what is going on in software development and therefore also in testing. Further, Cynefin seems to be an efficient enabler to create extensions to the old thinking frameworks in the particular contexts in which we use it.

Cynefin and software testing

Software development is generally not following simple processes. Development is obviously a human, creative activity. Good software development seems to me to be much more like a series of innovations with the intention to enable someone doing things in better ways.

Testing should follows that.

But if language limits us to different types of linear and causal thinking, we will always be missing that there is generally no simple, algorithmic or even causal connection between the stages of (1) understanding a new testing problem, (2) coming up with ideas, and (3) choosing solutions which are effective, socially acceptable, possible to perform, and safe and useful.

Experienced testers know this, but knowledge is often not enough.

James Christie added in his comments to the early draft mentioned above that as testers, with Cynefin we can better justify our skepticisms about inappropriate and simplistic approaches. Cynefin can make it less likely that we will be accused of applying subjective personal judgment.

I would like to add that the extended conceptual framework which Cynefin enables with us and our teams and stakeholders further more allow us to discover new and better approaches to problem solving

David Snowden on Cynefin

This video is a very good, quick introduction to Cynefin. Listen to David Snowden himself explain it:

 

AI personally found this article from 2003 a very good introduction to Cynefin:

The new dynamics of strategy: Sense-making in a complex and complicated world (liked page contains a link to download the article)

 

Efter 15 år som freelancer tør jeg godt tvivle på mig selv

I disse dage er det 15 år siden jeg tog springet og gik freelance. Det har jeg ikke fortrudt!

Jeg er blevet hyret ind som eksperten, der skal gøre det komplicerede enkelt og løse problemer. For det meste I lange kontrakter, men altid som den frie fugl. Jeg elsker det faktisk!

Konsulentjobbet kræver masser af kærlighed: Kærlighed til problemerne, der skal løses og kærlighed til de mennesker som har problemer. Ja, og kunden. Der følger også mere kedelige ting med: Kontrakter, fakturering,… den slags. De er en del af gamet.

I gamet er også en forventning om performance: At vi hurtigt kan gå ind og “levere varen” – uden slinger i valsen.

Ydmyghed er faktisk utrolig afgørende. For, – hånden på hjertet – konsulenter er langt fra perfekte og slet ikke ufejlbarlige.

Specialistrollen og forventningen om den sikre performance må aldrig komme til at betyde, at man ender med næsen i sky. Jeg kan godt blive lidt flov, hvis jeg ind imellem møder en anden konsulent med en attitude i retning af at de er universaleksperter, der altid ved bedst.

Jeg synes jeg selv er rimeligt god til at undgå den attitude. For mig hjælper det at jeg jævnligt mindes om nogle af de fejl jeg har begået. Efter 15 år i rollen har jeg ikke længere tal på, hvor tit jeg har fejlet i en opgave. Pinligt, men sandt. Og nu har jeg sagt det!

Den klassiske pinlige situation for mig som tester er et ”bugslip”: Kunden vil gerne have testet, at systemet vi arbejder med viser netop et bestemt resultat og jeg er hyret ind til at dokumentere kvaliteten af systemet inden vi går i produktion med det.

Jeg er testekspert og har indsigt i teknikken og projektet. Jeg udfører ordren. Det ser fint ud. Vi overholder planen. Alt er godt.

Men så kommer der melding om en fejl i produktion, og endda et åbenlyst problem som jeg simpelthen overså da jeg testede.

I sådan en situation er det ikke rart, at være i mine sko. Puh, jeg husker hver eneste gang det er sket, og det er mere end en gang! Det ligger desværre i testerjobbet, at det sker. Jeg prøver, at afstemme forventningerne om det, men sjovt er det aldrig.

Den situation og andre fejl jeg har haft del i har lært mig, at nok er det ret vigtigt at bruge sin erfaring og ekspertise, men det er også vigtigt at kunne tvivle på sig selv. Ja, tvivle: At vide, at ekspertise tit er langt fra nok til at garantere succes.

Sommetider er det faktisk netop ekspertisen, der står i vejen for at man gør det godt.

En generel ting jeg har tænkt lidt over (men ikke tænkt færdig) er, at vi alle faktisk burde blive bedre til at improvisere. Altså improvisere ærligt og blive dygtige til det: Fejle kontrolleret, observere det vi kan lære – og gøre bedre, fejle lidt mindre, evaluere, gøre det meget bedre.

Altså blive bedre til at undgå at lade os blænde af tidligere gode erfaringer – og derfor misse det åbenlyse.

Jeg tror i alle tilfælde på, at det er en kvalitet, når jeg som konsulent tager tvivlen med på arbejde – som en god ven, der hjælper til at jeg gør mit bedste. Og jeg tror på, at det er en kvalitet, hvis jeg deler tvivlen på en konstruktiv måde, så vi i fællesskab kan bruge den til at gøre vores bedste.

Ekspertisen og erfaringen er stadig vigtig. Men tvivlen må vi alig glemme.

I øvrigt føler jeg mig klar til at tage 15 år mere. Måske ses vi derude! Og bliv ikke overrasket, hvis jeg er eksperten, der tvivler.

The Art of Doubting

As a software tester, it is my job to question things. Questioning involves doubt, but is that doubt of a certain kind? Perhaps; let’s call it ‘good doubt’.

Monday May 15th 2017, I facilitated a philosophical, protreptic salon in Copenhagen about the art of doubting. The protreptic is a dialogue or conversation which has the objective of making us aware and connecting us to personal and shared values.

Doubt is interesting for many reasons. Self-doubt is probably something we all have and can relate to. But there seems to be value in a different kind of doubt than that with which we doubt ourselves.

Doubt is related to certainty. Confidence can be calculated statistically, and that seems to be the opposite of doubt.

Science almost depends on doubt: Even the firmest scientific knowledge is rooted in someone formulating a hypothesis and proving it by doubting it and attempting to prove it wrong.

Even religion, faith, seems to be related to doubt.

It is always interesting to examine the origins of a worud. The Danish and German words “tvivl” and “Zweifel” have the same meaning as the English doubt and all relate to the duo; two; zwei; to.

That appears to indicate that when we doubt we can be in “two minds”, so to speak.

So is doubt a special type of reflection, “System-2”, or slow thinking?

The protreptic is always about the general in terms of the personal. We examine our relations to doubt.

“What is it that our doubts wants or desires for us?” was one of my protreptic questions during the salon.

We circled a lot around that particular question. Finding an answer was difficult and we came back to self-doubt, which can be difficult to live with. Self-doubt can even harm our images, both the external ones and those that are internal to ourselves.

Leaders are usually expected not to have self-doubt: A prime minister risk loosing the next election if he doubts his own decisions and qualities. A CEO that doubts her own actions will drive the share value of the company down.

But there is a good doubt, and good doubt seems to be of a helpful nature.

Good leadership requires having the courage to doubt. It seems to help us act wisely and based on our experiences.

During the salon, my personal image of doubt changed. In the beginning I thought of doubt as a kind of cognitive function, perhaps a process I had to go through. Doubting could even be an event.

But at the end of the salon, my image of doubt changed into that a good friend walking with me through my life. Continuously present, if I want him.

With that image we found an answer to the question: Doubt is my friend. A friend who wants my actions to be driven not only by my instincts or simple gut feelings. A friends that help me shape my actions by my values.

dsc_4588_jpeg-srgb

Chaos to Kairos – NYC May 1st session on playful testing skills with music and philosophy

Jessica Ingrassellino and I will perform a workshop at the NYC Testers Meetup on Monday May 1st during the Test Leadership Congress. Join the meetup to participate.

The session will be based on the workshop we did at CounterPlay, an international play festival which took place in March in Aarhus, Denmark. Titled “Playful Software Exploration” the topic was value driven improvisation skills in testing. Together with the participants we tested, performed music and formed a philosophical, protreptic circle

The somewhat disturbing background of the workshop is that in a performance oriented and individualized tech industry, we are expected to drive ourselves to be the best in a complex or even chaotic reality. Remaining true to our professional and personal values while staying sane and ready to act and perform every day can be very challenging.

Our CounterPlay workshop was a success. Collaboratively we gained sense of and got to the core of important values in testing. We were even interviewed for the popular show “The Harddisk” on Danish national radio.

This time we would like to playfully explore the significance of Kairos in testing.

Kairos is Greek and means the supreme moment in which the future is transformed to the past with a particularly fruitful outcome. Kairos is important in rhetorics because while there are rules of good communication, there are moments in which speaking and acting is particularly fruitful: There is a time and space for the good talk. And even the best performance will fail if kairos is not considered.

This is an aspect of all improvisation and play, and good testing is in many ways always an improvised, playful act.

We know it when we perform exploratory testing.

But even when testing is turned into a controlled and scripted process, it makes sense to perceive testing in the microscope as a playful exploration and experimentation with potential and actual outcomes – even outcomes beyond the directly observable testing results: E.g. learning points for developers and management.

At the core is that testing makes a difference for people around us, even those who are not directly involved in testing and developing.

So let’s think beyond the processes, and functional and technical perspectives on testing, and explore software testing as a playful and human event with potential to create order in due time.

No prior knowledge or talents are required to join the workshop. But bring curiosity about values in testing, and be ready to play and improvise, introspect and think and reflect abstractly.

Best,

Jess and Anders

Kunsten at tvivle: Proptreptisk samtalesalon nr. 8

Det er videnskaben, der bærer vores samfund i dag, og med den følger søgen efter forståelse. Med forståelse kan vi skabe nyt og blive rigere og sundere.

Forståelse indebærer, at vi indtager et standpunkt, altså står ét bestemt sted. Dermed stiller vi os i modsætning til andre standpunkter: Forstår vi noget, kan vi ikke indtage flere modstridende standpunkter om det, vi forstår.
Så med forståelse følger holdning.

Holdning baseret på forståelse virker vigtigere end noget andet, når vi skal tage store beslutninger. Hvordan mon statsministeren eller præsidenten tænker, når han skal sende sit land i krig: Tvivler han og ryster på hånden? Eller har han læst og forstået situationen eksakt og præcist og udmønter derefter forståelsen i handlekraft?

Visheden, der følger med forståelsen, giver sikkerhed og signalerer styrke. Dens modstykke, tvivlen, kan være tegn på uduelighed og svaghed.

Men måske rummer tvivlen alligevel noget godt?

Mange krige har i eftertidens dom vist sig meningsløse og måske mest baseret på rygter og løgne. Vi ved det egentlig godt: Tit søger folk jo ukritisk efter vished og giver måske sig selv og andre indtryk af sikkerhed og styrke men snyder i sidste ende kun sig selv og andre.

Måske findes der en værdifuld, reflekterende tvivl, en dyd, som vi som ledere og mennesker bør kende og opdyrke for at bevare og udvikle kritisk sans og en sans for de sandheder, der rummer nuancer, der ikke kan måles videnskabeligt og sikkert?

Det kræver mod at tvivle, men tvivlen må i alle tilfælde være nødvendig, for at vi personligt og organisatorisk kan mestre de irrationelle psykologiske for-forståelser og mavereaktioner, som alle mennesker trækkes med.

Ja måske er tvivlen ligefrem en forudsætning for at kunne handle empatisk og begavet i det daglige?

I denne ottende samtalesalon vil vi derfor lede efter værdier i og omkring tvivlen, ja dyrke selve kunsten at tvivle.

Tilmelding er obligatorisk – og det er afmelding i øvrigt også, hvis du bliver forhindret. Hvis vi har tomme stole, vil vi nemlig gerne kunne give dem til andre interesserede.

Vi glæder os til at se dig!

Dato: 15. maj 2017
Tidspunkt: 16.00 – 18.30
Sted: Gjesing Coaching, Prinsesse Charlottesgade 31, kld, 2200 København N
Tilmelding til: karengjesing@privat.dk (obligatorisk af hensyn til forplejning)

Kærlige hilsner

Karen og Anders