Overview
- For a mature software company (5 years older or more), good unit test coverage is 40-60%.
- Good test coverage is higher for organizations serving Health and Safety verticals.
- Companies should calibrate their specific unit test coverage goals in light of customer feedback on code quality, maintenance budget, the presence of other testing methods, and for proof of concept/ “beta” repositories.
Unit Test Coverage Standards
General
| Age of Organization | ||
Unit test coverage | 0-2 years | 3-5 years | 5+ years |
0-20% | Good | Low | Very low |
20-40% | Good | Good | Low |
40-60% | Likely excessive | Good | Good |
60-80% | Likely excessive | Likely excessive | Very good |
80-100% | Likely excessive | Likely excessive | Likely excessive |
Health and Safety Organizations
Health and Safety organizations create software that can impact the physical well-being of individuals- such as medical software, transportation/ traffic control software. Testing standards are higher for these types of organizations.
| Age of Organization | ||
Unit test coverage | 0-2 years | 3-5 years | 5+ years |
0-20% | Very low | Very low | Very low |
20-40% | Very low | Very low | Very low |
40-60% | Low | Low | Low |
60-80% | Good | Low | Low |
80-100% | Good | Good | Good |
Implementation guide
- Determine which grid applies- “General” or “Health and Safety.”
- Evaluate whether these standards apply to the particular organization or should be adjusted. Factors to consider:
- The presence and quality of another method to measure code quality other than unit testing
- Customer/user perceived code quality, measured by customer satisfaction ratings, net retention, frequency and severity of bugs.
- If the unit test standards are overridden, the rationale should be codified and results reviewed quarterly.
- Once set, the code should be reviewed to see if there are any subsections of the code (at the repository level or within repositories) that require more or less testing.
- Available budget for creating unit tests should not be a factor in setting optimal unit test coverage. However, an organization may decide to accept lower
Notes
- The age of the repository/ application is not the determining factor, it is the age of the organization. Customers and users will expect that all of its applications should meet code quality (unit test coverage) standards based on the organization’s maturity, unless the application is explicitly noted as a Beta/ Proof of Concept product.
- It is possible to have too much test coverage. Development organizations should have crisp and compelling explanations why the standards are exceeded, as developer time spent on unit test writing takes away time from feature/ functionality development. One compelling rationale occurs when the programming language used makes test writing particularly easy/ fast.