Smart monkey testing

Andrei Terechko coordinated software testing and test infrastructure development for the Pareon dynamic analysis tools.

Leestijd: 5 minuten

Random testing tools have been state-of-the-art for decades in hardware verification, while in software they are just gaining momentum. Andrei Terechko discusses his years of production experience with random testing on software.

In daily practice software engineers focus on validating the code’s correct operation: the so-called ‘happy path’. Happy-path testing does a good job at achieving high line coverage, that is, the number of source code lines exercised by the test. However, many long code paths through multiple source files and libraries normally triggered by customers remain uncovered. And how about testing the code that checks input data validity? Even experienced test engineers often underestimate the importance of thoroughly testing error handling and error recovery code.

Various testing tools can create new test cases for your software. Tools based on formal methods, for example, can prove that certain invariants do not hold for your code and can provide a test case that triggers the assertion. However, they don’t scale to handling millions of lines of source code. Other tools merely examine a function’s input and output and generate a trivial test code template that misses its actual functionality. It saves typing, but not thinking.

This article is exclusively available to premium members of Bits&Chips. Already a premium member? Please log in. Not yet a premium member? Become one and enjoy all the benefits.


Related content