Wrox Home  
Search
Adam Kolawa - Co-Founder and CEO of Parasoft
Adam Kolawa is the co-founder and CEO of Parasoft. Kolawa, co-author of Bullet-
proofing Web Applications
(Wiley, 2001), has contributed to and written hundreds of commentary pieces and technical articles for publications, such as The Wall Street Journal, CIO, Computerworld, Dr. Dobb's Journal, and IEEE Computer. He has also authored numerous scientific papers on physics and parallel processing.

Articles by Adam Kolawa

Making Your Bug Tracking System Work for You

By Adam Kolawa, Co-Founder of Parasoft

Most commonly, a bug-tracking system, such as Bugzilla or GNATS, is used as a forum that enables the development and quality assurance teams to communicate with one another about product problems. This usually results in a system that is flooded with information about simple bugs that can be detected by the test suite. I believe this approach is misguided.

I have found that a bug-tracking system is most valuable when it is used in two main ways. First, it should be used to record and track all errors not detected by your test suite Careful recording of every bug report into the system facilitates problem tracking and provides valuable data about the types of errors teams or developers tend to make - data that can be used to hone error-prevention and error detection efforts. Ideally, the system ensures that the appropriate people are automatically notified about the problem, and it correlates bugs to source versions. Second, it should be used to record feature requests not yet being implemented. A reliable method for storing features facilitates the design phase of the next iteration. If recorded in this way, feature ideas can be easily and quickly recalled when it is time to debate the next feature set.

During development, use the bug-tracking systems to store feature requests, not track defects

When the product is in the development stage, developers ideally build test cases as they build code. They start by adding test cases that verify that each piece of code they add or change works correctly. These test cases are collected into a regression test suite that is run on a regular basis (preferably, every night). Each time a bug is found at this stage of the development process, a developer or QA team member develops a test case that verifies whether the bug is fixed, then adds this test case to the regression test suite. Test suites are thus constantly expanded and can be used to ensure that previously detected errors do not resurface as code is modified.

At this stage in the development process, the bug-tracking system should not be used to track defects. Test cases identify each error, so entering this same error information in the bug-tracking system would be superfluous. The bug-tracking system should be used only to store ideas at this point: Every time somebody has an idea for a new feature or for a better way to implement an existing feature, this idea should be entered into the bug-tracking system as a feature request.

After a code freeze, use the bug-tracking system to record both error reports and feature requests

After the product enters the beta-testing phase, source code is typically frozen. Because code in this stage cannot be immediately modified to respond to feedback, there is an added need for an information repository. This repository should now include both error reports and feature requests. Many companies try to enter only "real bugs" in their bug-tracking system, and soon find that there is a fine line between bugs and feature requests. While some issues (such as program crashes) are obviously bugs, classification of other "problems" is more difficult. For one person, a program lacking an expected GUI feature might be a high-priority bug; for another, it is just a feature request.

I recommend that instead of being petty and quibbling over definitions, you enter all feedback and change requests in the bug-tracking system so that you can easily access it later. For classification purposes, use "bug" to refer to a product issue that prevents a user from using the product as advertised, and call everything else a feature request.

Change requests should be recorded in the bug-tracking system throughout the code freeze stage - the time span from the initial code freeze to the day you start planning the next release. As you receive feedback, you can track the demand for each feature. The first time a feature is requested, you enter it in the bug-tracking system. Every subsequent time a user asks about that feature, you record another request for that feature. (You can store this information in any variable you like; for example, in Bugzilla, requests can be logged using the "vote" feature.) If you track requests in this manner, it will be easier to separate must-have features from eccentric ones.

During release planning, prioritize recorded feature requests and schedule their implementation

When you are ready to start designing another version of the product, you can leverage the stored information to improve your product. To start, pull all feature requests from the bug-tracking system, then start prioritizing them. Priority should reflect market feedback, including the number of requests you have received for this feature.

After you have settled on the priority in which the features should ideally be implemented, you start determining the order in which you are really going to implement them. If you cannot realistically implement a feature within the time allotted for the next release, that feature needs to be saved for future releases. You can determine the feasibility of implementing any given feature request by considering how its implementation would impact the existing code base. If implementation of a high-priority feature would require only minor changes to the code base, plan to add it to the next release. However, if implementation of your most highly demanded feature will require you to completely rewrite your code base, you have two choices: Save the feature for a later release, or change the release timeline so that it can be implemented.

The moment you decide to implement a feature request, remove the related entry from the bug-tracking system, create test cases that verify whether the feature is implemented thoroughly and functions correctly, and then add these test cases to the regression test suite.

After the feature is implemented, the regression test suite helps you determine whether it meets expectations and is ready to be passed on to customers.