Test Planning - Granularity

Hi,

I received feedback from my previous post and unfortunately there was too much tl;dr. The blog loses it's purpose if no one's reading, so I'll try to write more concisely or if the writing flow takes me, to divide the post into different subposts. And this is one of those.

Test planning as a whole is just too big to be fitted into one post so I just write about one of my favourite portions of it, the granularity. As Wikipedia puts it

"Granularity is the extent to which a system is broken down into small parts, either the system itself or its description or observation."

Test strategies and plan documents are within a scope of this post, but I'll leave them alone for now. They are actually really easy to produce with all the templates from ISEB, ISTQB, TMap, ISO9126, etc. The logic dividation of the subject/system to be tested and planning is the hard part. Every test designer wrestles with this problem; In what level of detail do I plan?

I'm not actually referring to test cases now. Some might think them as a curse word, because they usually tend to narrow testers minds, but from my point of view the term is irrelevant. You might use term set, scenario, charter, story, etc. I don't care. It's just dividation of the subject/system to be tested!

So let's take my previous post about community service, picking up trash. We didn't plan anything when we did that, but let's do so now. The purpose of this plan is to create organized way to clean trash from our environment and target audience could be anyone (it's a public blog, y'know ;). Let's start from the highest level of planning...

The Earth. Even when scoping out the Moon, our solar system, the galaxy, the know universe and beyond, it's quite ambitious, huh?

So, this is a bit too high level plan for us to work with. Let's divide.

Europe. Still a bit too rough to be used efficiently. So let's divide.

Finland. The graceful figure of Finnish Maiden. Dispite of all the gracefulness the level is still too high. Thus dividation!

Helsinki. The capital of Finland and my hometown. We start to enter to level which is controllable by a large team, which is in real life a sanitation department working under the city of Helsinki. But this level of planning only shows which area can be controlled by one entity, where as actual work it doesn't really help. So let's divide some more.

 

A part of Helsinki called Lauttasaari. For me this is the best place in Helsinki. Home. And we're entering the level which is actually tangible work-wise. One good person could organize trash picking with this level of planning. Good people don't grow on trees so let's go one step closer...

Ok. The final level of planning: Vattuniemi, which is southern part of Lauttasaari. One third of land surface area of the whole island. This can be executed by one person, not matter how good or bad, no matter what kind of information has been given. This is the idiot level. If you cannot handle this, go home.

So we've come a long way from the planet level to a southern part of an island. Let's get mathematical! Lauttasaari has 3,75 km² of land surface area, so my area of control, Vattuniemi, has 1,25 km². The Earth has 148940000 km². So basically if I find 1 kg of trash from Vattuniemi, there's 119152000 (around 12 million) kgs of trash on the land surface of the planet Earth. Similar estimation is done when calculating crowd on big events or animals in big migrations. It's all estimation, not exact science.

But now it gets tricky; Trash as bugs are not spread evenly on divided segments. According to Pareto principle 80% of the bugs are in 20% of the code. Therefore you must have coverage. Let's start this by putting it all to one logic structure:

You have six different levels of structure. With the same calculation as above, you have about 12 million branches on sixth level, which is quite a lot. If you are in this alone, all you have to do is to present this fact to stakeholders and you are either relieved from you work or put in charge of organizing the whole thing. And it's really nothing but putting people on each bubble; 12 million workers require corresponding amount of people who run the show on each of the higher five levels, whether they are coordinators, managers, directors or whatever. There are of course different kind of teachings and theories about organizing, team forming, etc. but as I'm not that academic person, this is how I reason the whole thing.

This the simple dividation of subjects themselves, but there can be also interfaces between these subjects, different kind of entry/exit data, different states, etc. and all of those create more demands on planning. And as many there are moving parts, people included, there are also at least as many possible breaking parts. The real world is always more wondrous than you could ever imagine.

Ok, I didn't really plan any exact scenarios. No guidance for picking up the trash? No action to run the sofware and no expected result? Why? Because it's not needed. When you're reporting about testing, you're reporting on the progress and about what works/doesn't work. There can be some supplementary reporting about defects, run results (e.g. performance), etc. but only in defect troubleshooting you are asked what you did exactly and that is done to bug reports. If you don't know what you should test, don't call yourself test professional.

Of course we are different, and have different tendecies when doing testing. If I really strain my brain towards neutral thinking, I might list some pros and cons of detailed/high level planning.

Detailed planning

• Easy to execute by others.
• A part of system documentation.
• Can be used as training material.
• Requires a lot of time.
• May limit testing, especially with experienced testers.

High level planning

• It's cheap! \o/
• Testing begins quickly.
• Allows new options and creative testing.
• Requires test experience.
• Not necessary repeatable after a long period of time.

Ther might be more of these, so please comment.

Whew that was tough! Let's get back on track!

Planning is ultimately waste, which often serves only testing itself. Traditional ways of testing aim for minimizing the test execution time by adding all kinds of overhead tasks, and planning is one of those. Modern world (e.g. Agile) instead tries to maximize the execution time either by reducing the overhead or by organizing it to be run in parallel with the actual work. Planning should be done to a level where it adds value. Quality over quantity. Plans that aim for finding bugs should be harnessed to benefit development, not testing.

James Bach wrote a presentation The Case Against Test Cases. There's a lot of sense in that, but as I've tried to tell in this post (I don't know if I've succeeded ;), I'd rather concetrate on the definition of granularity to build structure and support it with all kinds of means to test (test techniques, focusing/de-focusing, creative thinking, exploring, heuristics, etc.)

This is how I do test planning.

The final words I've taken from the opening credits of one of my favourite TV-shows, Star Wars - The Clone Wars:

"A plan is only as good as those who see it through."

Yours truly,

Sami "Padawan" Söderblom