Advantages of formal unit testing over just writing a lot of examples?
r, unit-testing
Solution
With a unit test framework you can test all kinds of things that you may not want to expose as examples to the end user:
- Does the function throw the correct error with incorrect input? A unit test framework will catch the error, thus passing the test.
- Unit testing should aim to get really high coverage of the code. You may want to test all of the combinations of all of the input arguments.
Another advantage of unit test frameworks is speed:
- You can selectively run tests in a given file, for example. In contrast, depending on package output means you need to build and check the entire package.
- This leads to much quicker identification of errors and a quicker development cycle.
My typical packages would contain dozens, if not hundreds of tests, but only a few examples that really demonstrate what the package is about.
In summary, I use tests to test, and examples to educate and help.
Problem
R has three well-developed unit testing packages, `RUnit`, `svUnit`, and `testthat`. Base R has examples built into its package functionality, which will return an error if they do not parse properly. The opinion of those whom I trust is that unit testing is better than writing lots of examples, but I can't quite put my finger on any specific functionality in unit tests that can't be replicated in examples. What features of using a unit testing framework in R make it superior to the ad hoc equivalent using package examples? For those not from the R world, note that examples for every function in a package are run each time the package is built, and the programmer is made to suffer for any warnings or errors.