We’ve all been there, that annoying website that doesn’t allow you to submit a form, or this app that keeps crashing. In short: bugs. In the software development world, we accept that it is generally impossible to write code that contains no bugs. All programmers are human, so they all make mistakes. Usually these human errors are combated by testing software.
There are two broad categories of software testing: manually and automated. When manually testing someone defines a set of test cases, each representing a list of steps that should be checked for problems. They then execute these cases by hand and report any unexpected behavior. Most kinds of automated tests work in the same way: someone comes up with a reasonable set of tests, but then implements them in code so they can be run automatically.
In general the following subdivision of software tests exists:
- Unit tests. These are tests on tiny parts of the code that do not depend on anything. They are generally quick and easy to write, but you need a lot of them.
- Integration tests. These test logical groups of units and make sure that they are “linked up” correctly. They are slower than unit tests, generally still easy to write and fewer are needed.
- End to End test. These test the entire software system from one end to the other. They are the slowest tests because they have many dependencies and as such are generally used sparingly. These tests often involve tools that can simulate user input, for example by automatically clicking and typing in a browser.
Automating these tests sounds great in theory, because they can be run whenever you want, without needing to wait for someone to do it manually. However, depending on the type of test, it might be hard to capture them in code, costing a lot of time. That’s why in some cases, the decision to write automated tests is a hard one.
However, in most cases automating all tests should be an easy choice. It is likely save a lot of time and money in the long run, because of some obvious and some less obvious reasons.
Higher developer productivity
In a survey among industry practitioners, 89% agreed that the cost of automated testing compared to manual testing is higher in the beginning, but automated testing leads to increased productivity over time. This happens because developers have to wait less often and for less time for tests to complete. They can know immediately if a change to the code broke something elsewhere. Since software is usually developed over long periods of time, the cumulative time and costs savings can be immense.
Bugs often arise because a change to the code somewhere has unexpected consequences elsewhere. When an automated test suite with good coverage exists, programmers do not have to worry about this. They can focus on moving quickly and developing new features, instead of treading carefully and acting defensively. Many software development teams run their automated tests on a central system, for each code change. This enforces all tests to pass for each change and further increases confidence and speed as a result.
When programmers write automated tests, they are forced to think about their code twice. This often leads to new insights on the subject matter and can uncover potential bugs that the programmer did not anticipate before. One particular programming methodology, test driven development (TDD), requires programmers write tests before they write the actual code. This forces them to think about the consequences of their code beforehand. Not only does this lower the number of bugs, one study shows that it even increases programmer productivity.
As an additional benefit, tests act as a form of documentation. They describe what the code is supposed to do as opposed to what the code does. They contain what the programmer intended their code to do. Programming is an extremely collaborative activity and tests will help future programmers altering and extending the software to get a better image of the code quickly. This will increase the productivity boost automated tests provide even further.
Automated testing is becoming more and more standard practice in the software development community. Although the time and money needed to implement automated tests might appear to be prohibitive, they will start paying off quicker than you might think. It will not only lead to a higher quality product, but save time and money in the process.