Automation Tips - Foreword

Robots that make you breakfast. Flying cars for commuters. That's science fiction. But software can do anything. So, why not make software test software? The reasoning goes, if one computer can do the work of three million mathematicians using sticks and sand, then surely one computer is worth an army of human testers. Indeed, test automation is an exciting idea that holds great promise. But beware. Automating some of your testing might or might not be helpful. Automation can save time, speed development, extend your reach, and make your testing more effective. Or it can distract you and waste resources.

Your investment in test automation is valuable to the extent that it helps you achieve your mission. The role of testing is to gain information. What information is your automation providing?

Automation efforts have been spectacularly successful for some groups but have left others unhappy and frustrated. Some of the failing groups have deluded themselves and their management into thinking that the work they put into automation yielded something that was helpful.

Use automation when it advances the mission of testing. Evaluate your success at automation in terms of the extent to which it has helped you achieve your mission.

Design your tests first, before deciding which to automate. This prevents you from falling into the trap of automating tests that are easy to automate but weak at finding defects.

Design automated tests differently from manual test. Much of the power of automated testing comes from using a computer to do things a person cannot do. Look for opportunities, such as being able to repeat the same tests over thousands of different data files. This prevents you from falling into the trap of only automating tests from the existing (manual) test plans and missing the big opportunities for test automation. When designing manual tests, you aren't likely to consider tests that apply repetitive operations over thousands of files; it would simply be too much work.

These conflicting messages derive from two important things:
1) Automating without good test design may result in a lot of activity, but little value.
2) Designing tests without a good understanding of automation possibilities may overlook some of the most valuable opportunities for automation.

To reliably succeed, you must have good test designers and good automators contributing to the selection and design of your automated tests. That's easy for us to say. It's harder to make it happen.