Tool Selection and Implementation

There are many test activities which can be automated and test execution tools are not necessarily the first or only choice. Identify your test activities where tool support could be of benefit and prioritise the areas of most importance.

The fit with your test process may be more important than choosing the tool with the most features in deciding whether you need a tool, and which one you choose. The benefits of tools usually depend on a systematic and disciplined test process. If testing is chaotic, the tools may not be useful and may hinder testing. You must have a good process now, or recognise that your process must improve in parallel with tool implementation. The ease by which CAST tools can be implemented might be called ‘CAST readiness’.

Tools may have interesting features, but may not necessarily be available on your platforms. E.g. ‘works on 15 flavours of Unix, but not yours…’. Some tools, e.g. performance testing tools, require their own hardware, so the cost of procuring this hardware should be a consideration in your cost benefit analysis. If you already have tools, you may need to consider the level and usefulness of integration with other tools. E.g., you may want a test execution tool to integrate with your existing test management tool (or vice versa). Some vendors offer integrated toolkits, e.g. test execution, test management, performance-testing bundles. The integration between some tools may bring major benefits, in other cases, the level of integration is cosmetic only.

Once automation requirements are agreed, the selection process has 4 stages:
1. Creation of a candidate tool shortlist
2. Arrange demos
3. Evaluation(s) of selected tool(s)
4. Review and select tool.

Before making a commitment to implementing the tool across all projects, a pilot project is usually undertaken to ensure the benefits of using the tool can actually be achieved. The objectives of the pilot are to gain some experience in use of the tools, identify changes in the test process required and assess the actual costs and benefits of implementation. Roll out of the tool should be based on a successful result from the evaluation of the pilot. Roll-out normally requires strong commitment from tool users and new projects, as there is an initial overhead in using any tool in new projects.