Tuesday, December 27, 2011

Exploratory Testing Hands-On - Part I


Back in business and I have tons of stuff to write about! But with what should I start? Why not the subject that is dearest to me, namely Exploratory Testing? There are many interpretations of what ET is, different entities have written tons of articles, blog posts and even books what it is and what it is not, gurus are battling it out in seminars... all done to a point of nitpickering. All this just confuses me and I've decided not to bother myself with that nonsense. Of course I still read everything and try to learn about ET, but only to aim for results and become better at it.

So, it's time for my first hands-on blog post. I usually encourage people to think for themselves by not giving them everything to complete what I'm wishing them to complete. I will continue on that path, but now I try to give a bit more; I try to give a real life example of how I do Exploratory Testing AND with a system that many consider to be the worst thing ever happened to a field of software testing; That is of course HP Quality Center.

So, for those who don't know what ET actually is, I try to give my own interpretation of it. Respecting the forefathers of ET, Cem Kaner and James Bach, the allmighty Wikipedia, James Whittaker and many others I consider ET to be a simultaneous process of learning, test design, test execution and test result interpretation guided by tester's personal creativity and experience (and that is experience on everything, not just testing) AND testing where next action depends on the results of the previous action. That is the official Powerpoint explanation I give to people. Awfully checklist like, ain't it?

At this point nitpickers have compelling need to say something, but I recommend them to back off! I'll bite! :)

But as ET is more about thinking than following certain rules, I try to think how those people think who I consider to be the ultimate Exploratory Testers. Here's are some of those:

Dr. Gregory House came the first in mind when I thought about the ultimate Exploratory Tester. He does something, learns from that and based on that learning, does something more, learns, does more, learns, etc. Simple isn't it? Simple aside, Dr. House fiddles around with immense amount of data in his head. He knows pretty much everything there is to know about medicine and applies that to his doing. And as the actor Hugh Laurie might follow a script, Dr. House never does.

Sherlock Holmes often states: "When you've eliminated the impossible, whatever remains, however impropable, must be the truth." It's all about logical deduction that he applies in his work as a detective. He solves the unsolvable, finds the unfindable by removing the unnecessary. Being a professional isn't about doing a lot of things, but about doing the right things; This is the most important thing Mr. Holmes (or actually Sir Arthur Conan Doyle) has taught me.

Jeremy Clarkson is one of my all-time favourite TV characters. I greatly admire, and to some point follow his down-to-earth, tongue-in-cheek and fun-loving attitude towards life and work, even the most serious bits of it. He's also a brilliant example of a true Exploratory Tester; He loves testing. He's propably tested pretty much every car there is, always embracing the experience, feeling the ride and giving his honest input. He rates cars by giving a story about them, never ever by numbers. Of course some of his analysis is based on metrics like speed, acceleration, price, etc. but the final verdict is never done based on numbers as it shouldn't be in software testing either. Imagine explaining the brilliance of Bugatti Veyron via endless reports filled with graphs, numbers, yadi yadi yada OR via 10 minute story...

Gordon Ramsay is in all measures one of the greates chefs of all time, and also a great example of an ultimate Exploratory Tester. Actually making food is the best analogy for doing Exploratory Testing; The magic is in tasting. Tasting is what separates a McDonald's chef from Gordon Ramsay, Jamie Oliver, Wolfgang Puck, etc. You all have seen in TV how Mr. Ramsay operates when making food. He might have a recipe, but it doesn't dictate his work. Tasting does. He works on menu level, thinks what components go well with others, fiddles with taste and texture, adds, removes, combines and within an instant comes up with something wonderful. Rather than following a recipe he follows the thinking behind the recipe.

Rather than following a script follow a thinking behind it. Cool. 

There are of course many more, but these four I consider to have the biggest influence on me when doing Exploratory Testing. It's all about thinking and following it over anything else. More about thinking in my previous post We Like To Think.

Ok, time for the anything else part then i.e. the actual methods, tools, implementation and whatnot. I've learned from my mistakes (Exploratory Testing, yey!) and I'm going to divide this post into several parts to prevent the tl;dr effect. Considering the amount I've written now and the amount I've written as draft, there should be three parts overall; This thinking part, the method/tool part and the actual implementation part. But as in Exploratory Testing, the plan might change...

But let's wrap this up with a quote. Today's topic is thinking, so let's have something around that. This piece is from one of my favourite motivational authors and has helped me to revise my actions when doing regression testing, forming test data, creating plans, etc:
"The world is what you think of it, so think of it differently and your life will change." -Paul Arden
Yours truly,

Sami McSöderblom


  1. Couple quick comments on ultimate Exploratory Testers...
    1) They are all domain experts in their field, which (according to Bolton and Bach) is not mandatory requirement from ET tester. Then again, while you don't have to be domain expert, it always helps...
    2) House needs his research team, Holmes needs Dr. Watson, etc. to record what goes on in "test session" for the client.
    3) Gordon Ramsay has lot of "best practises" when it comes to running business. Simplify menus, redecorate (usually on simpler style than original), only buy enough ingredients for couple days and use local ingredients if they are available, etc. Many times, these are the things that will offer businesses chance for survival, but after you've seen it done in dozen places, it doesn't feel like exploratory searching for solutions.

  2. I also agree that Quality Center makes testing very clumsy, pedagogic and strict. It is basically designed for bigger companies and bigger company bosses who want to see nice - and standard-looking - testing reports. It has basically not that much to do with the actual quality of the software that has been actually tested.
    I have also ALWAYS wondered (even though I have been developing quite a lot of stuff for QC - with workflow scripting and by using the API) that why it is a fat (they say it's thin - I say it's fat) client -based solution which works only on IE and requires Windows (yoi can get it working in Linux, I recall, but this is a hack and not a supported solution). I think this is an awful design and it should be a "normal web-based" application since it's basically just a GUI with a database behind it.

    But enough of bad mouthing QC. I also critisize the "nerdy" excel - or other - testing reports. :) So, I have an idea: let's design an exploitary test management tool which is user-friendly for a "normal" user as well. No fat clients, no "works on IE9 only", just a simple PHP, CGI, JEE etc. app with PostGres, SQL Server or MySQL database under it. Simple and easy to use GUI with reporting features. It could also be an Excel + Access -based thingy but I think that this world needs a test management tool that would actually give add value for testing! :)

  3. Thanks for comments, guys!!

    @ Juha;

    Teams and colleagues are important sources of ideas when doing ET, detective work, practising medicine, making food, etc. One of the most important qualities of an ultimate Exploratory Tester is to utilize his/hers resources and teamwork is just that.

    With Mr. Ramsay example I tried to concentrate purely on making food, the exploratory execution. Doing business is supportive action, contrasted to e.g. quality assurance or even software development itself of which Exploratory Testing is only a tiny bit. On top of being a brilliant chef (or Exploratory Tester), Mr. Ramsay is also a brilliant CEO or whatever Head Honcho.

    The domain expertise is however the most fashinating part; Ok, Mr. Clarkson is obviously a domain experts, petrol-head in the very sense of the word, but are the rest really domain experts? Sherlock Holmes has a case which is actually the domain, which he doesn't know very well at the beginning but grows into eventually. Dr. House knows medicine, but doesn't really know the domain i.e. the patient and his/hers problem at first. And Mr. Ramsay knows food i.e. the incredients which eventually grow into a domain. From my point of view they all know the means to overcome the domain, as do the ultimate Exploratory Testers. I've started to write a post "Arranging Atoms" where this subject is handled in more detail. Basically by manipulating the building blocks we can become masters in pretty much anything, including domain. Pretty deep, and no wonder I haven't been able to finish that post... :D

    @ Nixon;

    QC doesn't have to be that cumbersome. There are many ways of using it and it really can be adjusted to ET, even without scripting. But using it as I'm planning on describing in this blog post series even more emphasizes it's bloatware nature; It has soooo many functions and behaviour of which many are remnants from the time when the world had only waterfalls. It however can be used lightly, you can attach mindmaps, excels, session notes, etc. to it and it doesn't have to act the main role in software testing. Then again it's too expensive to be put to supportive role...

    About that TM tool... Should we build a company, gentlemen? ;)

  4. I know that QC can be used in a very different and clever ways BUT it still doesn't remove the problem that it's a clumsy and quite sluggish fat-client monster which only works on Windows and IE and costs bananas. ;)

    I think that company idea might be possible if there would be someone who could run the management and money stuff. I'm just a technical thug who knows nothing about anything important such as business. :D

  5. Your shit is the hard one. Money and management is nothing compared to that. Few courses of industrial management in open university, outsourcing bookkeeping, law actions, etc. to India or China, where they cost peanuts...

    Perhaps this comment section is wrong place for this kind of discussion. So I'll call you and we'll have beers in 2012!