Breaking the boundaries of familiar

Heyhey!

It's about time I wrote a blog post. :) Once again there has been a turmoil of everything and this blog has gotten unfortunately low priority among this everything.

But enough about that. I'm boiling with ideas. One of them being the idea behind a tweet I wrote just a moment ago:

This pretty much culminates my values in testing at this moment. But I agree, it requires some explaining. So let's get to that explanation part then.

Let's start with the definition of "exploration". The Oxford Dictionary describes it as "the action of exploring an unfamiliar area". Ok, a bit recursive (exploration is exploring, duh), but the part "unfamiliar" resonates with me. Whether you're travelling into the far reaches of space or diving into the depths of oceans, you’re doing exploration. Contemplating your actions and searching your feelings are in the very same sense exploration. You're digging into the unfamiliar in order to find something, to make observations and hopefully learn something too.

But to get to the unfamiliar, you have to first know about the familiar and the boundaries of it. Where light ends the darkness starts, and so on. Exploration is something you do to the unfamiliar, but what you do to the familiar? Checking of course.

And now it gets tricky. The boundaries of familiar and unfamiliar are not known until someone visits them. In software testing this is a neverending dilemma. It's about test coverage and depth, both being equally important and both being hopelessly hard to reach. I tend to characterize testing as a universe because of this very fact. It's infinite, constantly changing and as far as I know, expanding too. And there are number theories about it, each being more bewildering than the other. You can go as far or as deep as you want, but you will never fully conquer it. You will never make it fully familiar.

Imagine this picture being the test object:

The white areas are familiar and the black ones are not. Note however that there's actually just different shades of grey. The world of testing is not black and white, but a spectrum.

On the left end there is the caricature of checking; Check lists, expected results, pass/fail criteria, machine-controlled regression sets, you name it. Attempt of full control on the familiar with algorithms i.e. defined solutions for defined problems. And on the right end there's the caricature of exploration; Going boldly where no man has gone before. ;) Instead of full control, emphasis is put on learning and exposing yourself to observations. When doing exploration, there's a high demand for heuristics i.e. arbitrary choices, educated quesses, rules of thumb, intuition, whatnot. I consider heuristics to be questions you ask from a test object and oracles help you to interpret the answer, which is only in rare occasions just 'yes' or 'no'. I might create a separate post for this, because it's too large topic to be fitted into this post.

Our everyday life as testers is somewhere in between these two extremes. Imagine finding a bug from an area that should not have any more bugs left. You were in checking mode, running for example unit test set, and you hit something unexpected. An average tester would report that, but an excellent tester switches on the exploratory mode. The bug is an indicator that the boundary of familiar has been found and it's time to go beyond that. It's time to dig deeper for more information, perhaps for the root cause that might cause a lot more bugs. Please note that the more you are in checking mode, the more surprising something unfamiliar such as newfound bugs feel like. Think about it.

At the beginning there is exploration. When all starts, the picture is pitch black. Heuristic barrage commences and step by step, moment by moment the darkness starts to yield. Algorithms start to take place and some of the work (and if you're not careful, thinking too) gets outsourced to machines. At this point it's best to understand that there's a fallacy in familiarity and therefore in checking. That's because the picture is moving. The context that made you confident that certain area is familiar is now gone. The data has changed, the environment is different, the users are using the software in ways you've never anticipated. You created an algorithm that is dependent on all the known and unknown conditions that existed at the moment you started to create that algorithm. That algorithm you just made to make sure that this momentary familiarity prevails has gone obsolete at the moment you pressed 'Enter'.

Ok, I'm being too hard on you. :) Not all of the picture moves. There's a place for checking and usage of algorithms as there's place for exploration and usage of heuristics. Both are needed in testing. Both are important. I rest my case:

"Checking is about familiar. Exploration is about the world beyond the boundaries of familiar. Both are needed in testing." -Sami Söderblom

Yours truly,

Space Monkey