Your project has released! Maybe this was the first version or iteration, or maybe it was the 10th. However, being an attentive software engineer, you’re not done yet. Now it’s time for the postmortem. So, you have your standard “what went right” and “what went wrong” meeting, but you think you might still be missing some important information that would help the next phase of your project. Try bug analysis.
What is a Bug Analysis?
Your bug database is one of those sources of useful information. One form of a bug analysis is taking the data from the bug database and “crunching” the classifications (some may call them areas or types) of bugs and discovering where the largest number of bugs are coming from. The idea is to understand the issues and implement processes to reduce those types of bugs in the next phase of the project.
Informal vs. Formal or Intensive
You will first need to determine how intensive you want the analysis. For the scenario that follows, the bug analysis was a less intensive or informal review.
In this scenario, we reviewed a random sample of ¼ of the bug database. A more formal review would include reviewing more data. Only one person from the QA team performed the analysis for the informal scenario. For a more formal review, you could include other reviewers such as an outside resource, like an independent auditor.
A Sample Methodology
For the scenario, the QA personnel reviewed each bug from the sample and put it into one of the following classifications:
- Requirements – bugs that are usually caused by non-existent, or unclear requirements
- Code – coding errors and code integration bugs
- Test – this includes incorrectly logged bugs, unclear functionality (which could also be a requirements bug), and issues that can’t be reproduced
- Suggestions – all of the suggestions and requests for new or improved features
You may have different classification systems, but most issues will break down similarly.
A Sample Finding
In this real life scenario, the analysis included 77 bugs. The number in each classification was as follows:
- Requirements: 10
- Code: 35
- Test: 14
- Suggestions: 18
The initial finding was that the development team was not performing code reviews for all of the features and that some developers were skipping or limiting unit testing. Included in the scope of the next project phase was a re-commitment to unit testing and code reviews.
If there were more requirements bugs, then the team would know to improve its requirements management processes for the next phase or project.
If more test bugs were discovered, then the team would have to determine whether it was getting the documentation and support it needed; if the team members were simply making too many mistakes and needed further training; or if the makeup of the team or its leadership had to be changed.
A large number of suggestion bugs might mean that the project wasn’t really meeting the needs of the customer. In that case you should again review the scope and features of the application with your customers. To learn more about software engineering solutions and bug analysis contact SPK.
