Just explore.

Here is the scenario. Recently, the lead in my team at the US came up with the idea of using exploratory testing as a means to unearth better defects in the product under test. Hence, my team at Bangalore were asked to do exploratory testing for half a day and report all defects they find in this process, expecting a significant turn-over than normal days. But, looking at the number of reported defects during the next day, she obviously got disappointed and immediately wrote up a mail to me expressing her concern (although, in gentle words!) and asking for my thoughts on whether to have such bug-hunts in the future. This was my reply to her.

NOTE: So, this is what. Sometimes, I get impressed by my own writing after I finish it up and want to show it up to the rest of the world. Part of the reason for this post to appear here is that, but partly I want to drive home the point that if you care about product quality and want to go bug-hunting, rely on exploratory testing. Everything else is of lesser priority.

"I am a great fan of exploratory testing.

This kind of testing allows to explore relatively 'unknown' areas using relatively 'un-tried' techniques and for this very reason, I strongly believe that this works best when we go on bug-hunting. Test cases are good, as far as they are looked upon as guidelines. However, what I've seen happening in the team during the past 3 months, is that repeated execution of them leads to boredom, errors and the result is missing defects and less productivity. More mechanical and less creative work will miss some great scenarios. Testers no more think like customers, and at this point the whole testing becomes another mundane, ineffective job.

Now, all this said, I think the team is receptive to this, but not set in the right mood for exploratory testing as yet. People are so used to the routine test case execution that their brain started rusting over the time. It will take some more (maybe, a lot more) time to exercise their minds before we can expect glorious achievements from our testing team. Nevertheless, I believe it is possible, with a lot more stress given to such exercises in the future. Encourage people to go all out to do exploratory testing as frequently as they can. While they engage in this sort of testing, let them take notes of what all they've done and discuss with the rest of the team at the end of the day. This will not only help individuals' defect counts, but also help team level communication that will ultimately end up in improving productivity. Just to clarify, I am by no means saying that test cases should be ignored. Test cases are the guidelines and should be seen as such. I am also not against test case execution since it rates the product's ability to work correctly using the 'knowns', but all I'm saying is that it is equally or even more important for us to know how the product performs using the 'unknowns'.

Now, to answer to your other question about the less number of defects filed yesterday, I didn't talk to the team in detail, but chatted with them for a few minutes. The feedback I got was that the product is in a much stable state now and hence the defects are less. It could be right (in which case, we all ought to be very happy), but I tend to think bugs are still hiding in the product very much, but just that people are not capable enough to exploit the 'power to explore' fully. So, I am afraid, if we stop encouraging people to do more of this now (because of whatever reasons), then people will go back to their old self and start rusting their brains again and, in this process, we had only wasted half a day yesterday!

After all these days, a long mail from me, for a change:-) Now, you may differ in your opinion and you've all rights to! I just described, what I think we must practice to see a change, especially if you want to see people enjoy the testing...it's all based on my own experience

Expecting a good debate on the subject and some important decisions that will help our team grow to the next level...

Regards, Anand Iyer"