Bug Reports: life cycle

In keeping with the bug reporting theme, reader Bob was wondering what sort of life cycle a bug report goes through at REAL. Since I'm not part of the testing staff, my answer might be slightly different from what someone like Kyle would say. Also, since I'm a programmer, I figured I'd give a tongue-in-cheek example program (note: it's not a very OOP example!). It's a life cycle, which is just fancy "human talk" for "algorithm", right? ;-)

Sub Main()

  while not OutOfMoney

    report = GetIncomingReport()

    select case report.Type

    case Report.kFeatureRequest

      ProcessFeatureRequest( report )


    case Report.kBugReport

      ProcessBugReport( report )

    else

      HandleError( "Invalid report type" )

    end select

  wend

End Sub

Sub ProcessFeatureRequest( rep as Report )

  if EvaluateValidity( rep ) then

    FileReportForLater( rep )

  else

    CloseReport( rep )

  end if

End Sub

Sub ProcessBugReport( rep as Report )

  if CanReproduce( rep ) then

    AssignForEvaluation( rep )

  else

    CloseReport( rep )

  end if

End Sub

Sub AssignForEvaluation( rep as Report )

  if rep.Version = kCurrentVersion then

    Assign( rep.OwningEngineer, Report.kHighPriority, kCurrentVersion )

  else

    Assign( rep.ProperOwner, DeterminePriority( rep ), DetermineFixForVersion( rep ) )

  end if

End Sub

Function DeterminePriority( rep as Report ) as ReportPriority

  if rep.IsPrioritySupport then

    return Report.kHighPriority

  else if rep.IsCrashingOrDataLoss then

    return Report.kMediumHighPriority

  else if rep.IsCommonlyUsed then

     return Report.kMediumPriority

  else

     return Report.kLowPriority

  end if

End Function

Function DetermineFixForVersion( rep as Report ) as RelativeVersionNumber

  if rep.IsPrioritySupport then

    return Report.kVerySoon

  elseif rep.IsCrashingOrDataLoss then

    return Report.kVerySoon

  elseif rep.IsEasyToFix then

    return Report.kSoon

  elseif rep.IsCommonlyUsed then

    return Report.kSoonerOrLater

  else

    return Report.kSomeday

  end if

End Function

When a bug report comes in, it hits our Inbox. That's basically just a holding pen for any reports that have yet to be looked at, or haven't moved forward in the process. It's someone's job to read each and every report that comes in, so our Inbox rarely stagnates. The first hurdle that a report must jump is whether it's a feature request or a bug report. If the report is a feature request, then it gets evaluated to determine how "interesting" it is. If it's a feature request pertaining to a brand new feature (while the product is still in alpha or beta), then it usually gets assigned to whoever owns the feature for further evaluation. Otherwise, the request is usually assigned to a holding bin that we can look at when it comes time to determine schedules. If the feature has an obvious owner (such as compiler feature requests) it just goes onto someone's schedule for an undetermined release. Either way, it's almost always a "we'll look at this in more depth later" situation. However, one thing which we've been improving upon lately is communicating with customers about things we don't think we'll ever implement. It used to be that requests would go into the system to languish and die. The problem with doing that is: it's wrong, on many levels. It clutters up our feedback system, so it's harder to separate the wheat from the chaff. It provides the user with no clues as to whether we intend to implement the feature or not. It just sucks, and so we've stopped doing it. Now, if we don't think we'll implement your feature, we will tell you that when we close the report. What's more, we'll tell you *why* we don't want to implement it, because there can be many good reasons. Sometimes it's because the feature would cause code to break. Sometimes it's because the feature isn't in line with what we want REALbasic to be. Sometimes the feature is so incredibly huge that we couldn't possibly implement it. So if we close a feature request, we make sure to say why.

So let's say it's not a feature request, but is a bug report. In that case, the next step is to see if we can reproduce the behavior in house. If we're unable to reproduce it, then we close the report and ask the user for further assistance in reproducing the issue. But if we can reproduce the bug, then it gets kicked over to me for scheduling. I prioritize the reports, determine who is best to work on them, and figure out what release they should be worked on. Basically, it depends entirely on the bug as to how that all works. For instance, if it's a newly-introduced bug then it gets a higher priority than one which has been around for ages. If it's a data loss bug, then it's higher than a cosmetic issue. If the bug is very simple to fix, it's scheduled sooner than one which will take a significant refactoring, etc. Another thing which happens at this stage is that sometimes bugs are closed as "won't fix" or "by design." If a report comes in that's behaving as expected, sometimes testing doesn't catch that fact (they just see the user say "when I do X, Y happens" and can reproduce the issue). Also, some bugs are things we won't fix -- they're not by design, but they're also not going to get fixed. A good example of this would be bugs reported against deprecated functionality. Again, if the report gets closed at this stage, then I make sure to put in a personal message explaining why I've closed it.

That's the day to day process that bugs go through, but there is still a bit more to talk about in terms of their life cycle. At release boundaries (as one release is finishing up, and another one is about to begin), we go through all of the bug reports and feature requests which haven't been assigned for a specific release and start to dole them out. (Part of this is also rolling forward bugs which haven't been fixed for the current release that have been assigned.) Basically, we start dishing out schedule fillers and checking to see what FogBugz has to say in terms of likelihood of meeting deadlines.

Another thing I want to talk about is what I mean by "close report." In the old feedback system, this was a way to finalize a report. "Closed" meant "dead, gone, never coming back unless the moons align properly." That's not the case at all in the new system! Closed means "please reply to automatically reopen the report." It's basically a way for us to do housekeeping. We close a report if we feel there's nothing else that can be done unless we get more information from the reporter. So if the reporter never responds, then there's nothing more we could have done anyways. But if the reporter does respond, then we might have something else to do -- and the report may rejoin the process. So "closed" isn't a bad thing, I promise.

Of course, this is still a pretty high-level explanation of how bug reports are handled. But it should give you a good idea as to how reports are handled.