2009年1月7日星期三

Expand your reach instead of trying to repeat the same tests over and over

Use automation to expand your reach and extend your senses, allowing you to see more and do more.

You just can't run some tests without automation. You can run others one a much larger scale. Here are some examples:
Load Test. What happens when 200 people try to use your software at the same time? What about 2000? What about 20000? You'll need automation to simulate these scenarios.
Performance Benchmark. Is system performance getting better or worse? You can instrument automated tests to capture time measurements each time you run them. By collecting these measurements and reviewing them as a tune series, you can detect performance degradations. Use the same approach to benchmark uses of resources, such as memory or storage.
Configuration Test. Software often must work on different platforms, in different configurations, attached to different peripherals. How do you cover them all? Automation helps you increase your coverage. To make this work, you must ensure that your tests are portable across platforms.
Endurance Test. What will happen when your product has been in use for weeks or months? Memory leaks, stack corruption, wild pointers, and similar error may not be apparent when they occur but will eventually cause trouble. One strategy is to run a series of test cases over a long period C days or weeks C without resetting the system. This requires automation.
Race Conditions. Some problems occur only in specific timing situations. The coincidental timing of two threads or processes contending for the same resource results in an error known as a race condition. These are often hard to find and hard to reproduce. Automation can be a big help because you can repeat tests with many slightly different timing characteristics.
Combination Errors. Some errors involve the interaction of several features. Use automation to test huge numbers of complex tests, each of which uses several features in varying ways.

Those approaches focus on using automation to create new tests or to repeat product usage in ways designed to uncover new bugs. None of these tests is simple to implement. You may have to work up to them by automating different parts of the testing and developing tools to assist you Nonetheless, I think this is often a better goal for the automation efforts than repeating the same feature tests again and again.

没有评论:

发表评论