Exploratory Testing - An Agile Approach?

By Ray Claridge
Pin It

outside of the boxIn a previous post I touched upon the notion of developers helping with the testing bottleneck, by executing scripts predefined by the tester. However, this does come with a risk because one test approach that is difficult to handover is the exploratory side of things. Let me explain...

Since moving into agile development where the requirements are typically lightweight and evolve during the coding phase, its difficult to plan all test cases upfront. More often I'm finding myself having to use an exploratory approach to ensure as much coverage as possible.



Give a non tester a script to execute, and they're only going to cover the predefined tests and not be thinking outside of the box. Hence the increased risk.

So what's the answer to this? Ensure your test script coverage is sufficient? This would require your requirements to be a really low level and set in stone (which goes against the whole agile approach).

To be honest, I don't think there's an answer to this (apart from the tester executing all tests). However, improving your team's understanding is a starting point.

What is Exploratory Testing?
Exploratory testing is simultaneous test design, execution and learning.

How is that different from structured testing?
Structured testing, is designed to verify that the software under test meets the specified requirements. This is not the same as verifying the quality of the software. Testing is directed and constrained by the design specification and test plan, both of which may have been created before a line of code was written.
Exploratory testing is not constrained in this way. The objective is to expose information that is of value to stakeholders and assess the quality of the software, of which compliance with the specification is only one factor.

Emergent Behaviours
Exploratory testers are always looking for emergent behaviours, which are behaviours that could not have been predicted prior to the start of testing.
Structured testing focuses on expected behaviours, so emergent behaviours remain entirely untested. It should not be a surprise that this is often where the biggest bugs are found.

Learning
Run high level tests allows testers to learn about an aspect of the application's behaviour, and each test provides information that allows ever more powerful tests to be devised. This knowledge is invariably lost in structured testing projects.
Rather than ask does the application meet this requirement, ask questions like what happens if...

Adaptability
The exploratory tester starts with an outline test plan in mind, but can adapt it in response to changing contexts. If one part of an application is not yielding bugs, it may make sense to test in other areas instead. Alternatively the tester may consider using different test techniques in the same area.

Exploratory testing is an efficient and effective approach to testing software. It produces useful results from the first hours of testing, and finds the biggest bugs faster than any other approach. Furthermore, it finds bugs that other approaches won't find at all.

Ray

6 comments »

  • Kay said:  

    Hi Ray,
    I am an agile tester and I found the aricle interesting as it reflects my working environment. I have few queries on the above:
    1. Could you explain the 'emergent behaviours' in more detail?
    2. How do ascertain that exploratory testing finds bugs that other approaches won't find at all

    Thanks,
    Kay

  • Ray (tester troubles) said:  

    Hi Kay,

    1. When testing in a more traditional way, you plan for expected behaviour, so an emergent behaviour is the opposite. These are behaviours that make you divert from your plan.

    2. How many times have you run a planned test script over and over again without finding a defect only for one to appear after the softwares gone live? I guess this has happened. So by including an exploratory phase into your plan you're more likely to find defects that would otherwise miss.

    Hope this helps.

    Ray (tester troubles)

  • Anonymous said:  

    Hi Ray,
    I am new to Agile and found your articles very interesting. R&D in my company is currently using Agile and for testing we do mostly Exploratory testing. If possible, could you please post some templates for writing "Stories". Or any article on how to get started with Agile?

    Thank you,
    Pavana

  • Software Testing said:  

    I don't have knowledge about Agile. But, after reading this article I got some idea about Agile and I am going to implement the Agile in my organization soon.

  • oksana said:  

    Ray, thank you for an interesting article on Agile testing. I have a question for you about test planning and test execution reporting. I am working in an organization where waterfall methodology is adopted. Prior to SIT, test plan and test scripts have to be implemented, number of test cases is published, test execution is planned, and reporting is based on number of tests executed. If the requirements and thus, test scenarios are continiously emerging, how is the overall project status can be assessed? How can we know and effectively report to the management whether the project is on track or not? Thanks.

  • Regration testing said:  

    "Yet another interesting post Ray. Thank you for sharing it. I must say that your tips through posts helps QA testing experts to make a move from waterfall to agile. Keep them coming.
    "