About Working Memory and Testing

Cognition is the word used to describe all the mental processes happening in our brains. Cognition relies on working memory, which is the memory we are using when we are solving problems or performing tasks.

Testers, for example, often rely on working memory to keep track of what we are doing and the observations we are making while we test.

Working memory is fallible. You will probably have experienced going to the fridge to check something, yet when you get there, you find that you have forgotten what you were lookoing for. It usually helps returning to the place where you first thought about it, but sometimes it’s gone completely. It can be quite frustrating!

Adults can usually hold up to five things in working memory at the same time over a time span of several minutes. Small children can only hold one. A friend of mine, who used to work in a nursery, told me about a small boy, who had just learnt to walk. He was good at it, but when someone called his name, he fell on his behind. Every time! It’s was quite cute, she said.

Some people have very poor working memory. My 13 year old son Aksel has Asperger Syndrome and a severe attention deficit disorder (ADD). His working memory is very poor. In school, he usually looses focus on what he is doing after less than a minute, unless something or someone is helping him. By using a lot of mental energy, he can keep focusing over about 10 minutes, but it exhausts him and he has to take a break afterwards.

Yet, he can build the most fantastic Lego creations, particularly on the computer, and be concentrated about it for several hours. He is also excellent in a go cart, where he can stay 100% focused for more than 30 minutes – and is still completely relaxed afterwards. I’m usually exhausted after just 10 minutes!

So what’s the difference?

The difference, I beleive, is that the Lego creation and the go cart isn’t loading his working memory: He does not have to remember what he is doing as it’s in the context. The Lego building application is even helping him keeping all the bricks in order: He doesn’t have to focus on looking for that missing brick, but can quickly browse for any brick without loosing focus on the thing he is building.

At school, Aksel and his teachers are working hard to find strategies for off loading his working memory.

Aksel and I share a lot of genes, and though my working memory is far better than his, it isn’t quite as good as I would like it to be. I’ve learnt myself a few strategies to overcome it, and it’s usually not any big problem, though.

But since testing, and in particular explorative testing, is very demanding on working memory, I sometimes feel working memory impaired and I have developed a few strategies for myself to overcome it.

Test scripting is one of those strategies: Scripts are working memory aids for me.

Scripting by mind maps, however, doesn’t work well for me. The linear script is probably easier to navigate for me. (I love mind maps for taking notes and for manuscripts when I’m speaking, though.)

Scrips are also excellent providers of a ”safe home” when I’m diverting off to explore something: I don’t have to worry about forgetting what I was doing. I usually don’t even have take notes, which is very good, since note taking is something I’m not very good at (though, with discipline, I have improved over the years).

Developing strategies to assist your working memory can a good thing for anyone, even if you have excellent working memory, but there’s no one solution that works for everyone: Some like scripts, some hate them.

Experiment and stick with things you find work for you: Handwritten notes, mind maps, scripts, drawings. And if you are in the mood, try to use your body: Most of our brain cells are allocated to interacting with our muscles and senses, and they can work for you too: Count with your fingers, move yourself (and your laptop) around, talk to yourself about what you’re doing, etc.

In case you want to learn more about the concept of working memory, I can recommend the following book. It’s about working memory in children (for education), but the concept is the same for everyone:

Working Memory and Learning: A Practical Guide for Teachers

The problem with test documentation

The agile manifesto explicitly values ”working software over comprehensive documentation.” In test, this means that actual testing is valued over test documentation. I would have put it this way: Focus is on the quality of the product, not on the quality of the documentation of the product.

We can probably all agree that it’s more fun to test and explore software than writing documentation. But it would be too radical, if we skipped writing documentation at all!

I think, however, that we testers should be more critical about the documentation we actually do produce, and that we should look for ways to improve the documentation we’re making.

The problem with documentation is not that too much time is spent writing it instead of actually testing the product. The problem is that the quality of the documentation is often just not good enough.

Most organizations have standards for test documentation and require their testers to write specific types of documents based on mandatory templates.

Templates are generally a good thing, but they can be problematic if you limit your writing process to ”filling in the gaps.” A good document contains useful information to the reader, so the most important step in the writing process is finding out what information is actually needed.

Not all sections of a template can be equally useful in all contexts (or useful at all), and very often you need to document things which there has not been left space for in the template.

But, finding out what to document is not trivial. Basically, it requires that you know the context of the project you are working on. Determining context can be difficult if you are new on a project, but asking questions can give you answers which will help you define the context.

Here are some questions, which I find useful:

  • Who will be reading the document? How will they be reading it? The challenge is to include content which the actual readers will find useful and to structure the content in a way so that they can find that information.
  • What information are they looking for? Try to get people to answer in concrete terms rather than using abstract vocabulary. The problem with written documentation is that stake holders will often not read it – most testers seem to prefer to explore systems on their own rather than reading documents, and managers will often not have time to read everything, but will only check the headline and certain details. But if readers get what they look for, chances are that they will read the document.
  • What kind of analysis do I need to carry out on the system to better understand it and test it? Writing a document about the system can often help me understanding it. I will discover missing knowledge and knowledge which I didn’t see initially. The writing process is part of the analysis.
  • Are there regulatory requirements which specify that testing should be documented to a certain detail and in a particular way? In this case, the test documentation is actually a product of the project.
  • Should the document assist knowledge transfer once I’m no longer on the project? In that case, the big question is what kind of knowledge should be ”transferred.”

I checked the section about documentation in Kaner, Bach and Pettichord’s Lessons Learned in Software Testing a few days ago. They have a better and longer list of context-free questions to ask about documentation which will help you find out what kind of documentation you need. They also list a few references to other lists of useful questions, so I suggest you take a look at that too.

Are there any specific types of documentation which is particularly useful? Indeed, there is:

  • Mind maps are becoming increasingly popular with testers, but ‘old style’ software diagrams like swimlane diagrams, state diagrams, and flow charts etc are still working well. The problem with mind maps is that they are often personal to the author of the map and not very easy to read.
  • Test scripts can be very useful in initial stages of knowledge transfer: A script can help another tester finding out how a certain procedure is performed in the system. However, a script will by itself tell the tester anything about the context of the script, and this is something which is often missed: Knowledge about a system under test is much more than knowing how to do things.
  • Check lists are actually much more useful than the name implies: A check list will list things to verify, but unlike a script will not specify in detail how to verify them. That information has to be available elsewhere e.g. in user manuals.
  • I always look for a document describing the system context in a top-down manner: What is the system doing, for who, and how? If it isn’t there, I don’t mind writing that document myself.
  • A catalogue of tools used in testing is often also very useful. Test tools are often not well documented (or not documented at all), and that can be a hurdle for new testers when they come aboard a project. A well written ”tips and tricks for testing system X” will get them up to speed faster and can act as a platform for sharing knowledge about testing specific parts of the system. I like Word documents for their self-contained’ness, but a Wiki could be better in many situations – the important thing is that such a document is actively maintained.

What are your preferred test documents?