Saturday, January 19, 2013

Life cycle of a MySQL bug

I had already written once on how to report bugs to http://bugs.mysql.com to have high chances for the bug report to be noted and processed fast. But why this matters at all? Because chances are high that bug report will never be opened and read by any MySQL developer in Oracle while its status is just "Open"! This is a short answer.

If you are surprised (the story is different with, say, Percona Server bug reports, where just reported "New" bugs are often reviewed and commented on by many engineers) let me present more details on how bugs processing worked at least since 2005 when I joined MySQL AB and what important changes happened when MySQL became Oracle software.

I'd like to explain most important statuses for MySQL bug reports at bugs.mysql.com, to begin with (you can see them all and use for search). Here they are, with short explanations:
  • Open - this is a status of a bug that is just reported or had recently got a comment from reporter or any other user. Sometimes MySQL engineers can set status back to "Open" when they do not know how to proceed. 
  • Closed - Usually it means that the bug is fixed. You should expect to see a comment at the end of report with short description of the problem and fix, and all versions that had got the fix. Original bug reporter can set status for his bug to "Closed" if he considers the problem was not a result of any bug in MySQL software, found that the problem is already fixed, or just does not care about the bug any more.
  • Duplicate - it means that there is another bug report about the same problem. Usually you should see a comment with bug number for the other report in public (or Oracle internal) bugs database. This status is usually set by MySQL engineer who processes bugs or by developer who started to work on this report but recognized familiar case.
  • Analyzing - it means that somebody wants or started to work on this report. Usually you should see engineer name in the "Assigned to:" filed in this case. This status is often used as a semaphore to show other engineers that there is no need to work on this bug.
  • Verified - one of the most important values for community (only "Closed is more important). It means that report is accepted by MySQL engineer as a real bug that this engineer is able to repeat (or explain MySQL developers how to repeat or what is the problem). So, the bug is confirmed internally.
  • To be fixed later - it means that while the problem exists and verified, there is no way to fix it in current GA or development versions, or fix needs serious changes in software or data format, or serious development efforts and thus requires long term planning. Usually it means "To be fixed never".
  • Won't fix - developers just do not want to fix this. This is more a "feature" than a "bug", even if user thinks differently. Users often ask for some things that, while make sense for them and their use case, may be against SQL standard or have acceptable workarounds in frames of current implementation etc.
  • Can't repeat -  bug reporter had NOT provided a complete, repeatable test case or the problem is not repeatable with current code or anywhere outside reporter's environment. Sometimes (rarely) it means that MySQL engineers had not tried hard enough to reproduce the problem.
  • No Feedback -  when bugs spent 30 days in "Need Feedback" and had not got any comment from anybody, this status is set for it automatically. It means that original bug reporter does not care about the problem any more and is not going to reply to questions/requests made, and other community users also do not care enough to find the report and add missing detail. Reports in this status are usually ignored by everybody in MySQL Support and Development.
  • Need Feedback - bug reporter (or anybody else who cares) should provide additional information about the test case, expected results, environment or version used etc. Without this information MySQL engineer was not able to verify the bug or understand its impact and severity.
  • Not a Bug - the problem (if any) was a result of reporter's misunderstanding of the manual, the way MySQL server works, misconfiguration or other mistake that has nothing to do with any bugs in the code.
  • Unsupported - the problem reported happens only on unsupported OS (like HP-UX 11.10), with old and no longer supported MySQL software version (like MySQL server 3.23.58) or it happens on a custom build or fork (like Percona Server or MariaDB).
Note that I've used bold for some status values, I'd call them "final", usually any work on the bug report is completed when it is in final status. You can try to re-open bug in final status, send more comments there but usually the decision is final. Probably it makes sense to open new bug report and reference older one in "final" status there if you have anything useful to say or add, or can prove that the fix is not correct or not complete.

There are also other status values, generic ones (like "Active" - bugs that are not in final status), but they are just a short form for a set of status, and development-related values  (like "In progress", "QA review" or "Patch queued") that you can see in some bug reports but that became obsolete and not really used since Oracle forced MySQL development to use internal bugs database that is used for all Oracle software.

So, with these basic status values in mind, let me explain the life cycle of a typical MySQL bug report.

Bug starts as "Open" and from this status it can go to "Analyzing", "Duplicate", "Verified", "To be fixed later", "Can't repeat", "Need Feedback", "Not a Bug" or "Unsupported". You should expect some public comment explaining status change for all changed but to "Analyzing".

Bug with status "Analyzing" can become back "Open" (if analysis led to nothing useful or just had not happened) or it can become "Duplicate", "Verified", "To be fixed later", "Can't repeat", "Need Feedback", "Not a Bug" or "Unsupported" based on the analysis.

Bug with status "Verified" can (and should!) eventually become "Closed", when it is fixed. But it can end up in "Need Feedback" or "Can't repeat" (usually it means incomplete test case, probably a mistake of engineer who verified it), "Duplicate" (developers know better, based on code insights, what fix really solved or is going to solve the route cause of the problem), "To be fixed later", "Won't fix" or even "Not a Bug" (again, rarely, but MySQL engineers who do bugs processing can also misunderstand the manual or test case).

Now the most important thing you should know about MySQL bugs processing the way it is done now in Oracle. When bug is "Verified" and(!) considered serious enough, it is copied to the Oracle internal bugs database and further processing, including status changes etc, is done there. All further comments to the public bug report are then copied to internal bug report automatically, but no comments or status changes in internal bug report are copied to public bug in any other way but by explicit action of some Oracle engineer.

You will see the next status change for the public bug report only when it will get some internal status that corresponds to "final" status above and(!) somebody will care to change its status. Usually this care is taken by members of MySQL Documentation team when they document bug fixes for the release (or soon after that). So, "Closed" bugs will eventually be closed at http://bugs.mysql.com, there is a procedure for that. For any other final status I am not sure about any procedure applied (ask Oracle on how it is done now, my knowledge is outdated anyway). So, you should expect that public and internal bugs databases are out of sync. Community can not know what happens with the bug until it is closed or get any other final status. maybe nothing happens or active development happens. We will never know, until somebody will take care to explain in public!

Bugs that were NOT verified and copied just never appears in Oracle internal bugs database, and busy MySQL developers may note them only occasionally, or when somebody (like Oracle customer or colleague) will ping them personally, or will blog about them. They are probably not allowed formally to work on a bug report until it appears in internal bugs database (correct me if I am wrong).

Coming back to life cycle, bug in "Can't repeat" status can be re-opened by adding comment, but, if I remember it correct, this may not happen automatically, so some MySQL engineer have to note recent comment and check it. Theoretically if test case or missing detail is provided the report can end up then in any status that "Open" bug can get.

"No feedback" is a really bad status for a bug. It means that nobody cares. If you care, add a comment and bug will get status "Open" immediately. As far as I remember any user not only original bug report or MySQL engineer) can add comment (there were different policies over time on this). Now with Oracle SSO login needed to add comments and all that Captcha checks everywhere spam in bugs database should be rare issue anyway.

"Need feedback" is the most important status for community users who care - check bugs with this status and add your comments, similar use cases, missing details etc. Do not rely on original reporter if you understand the problem and question asked - reply immediately!

Final statuses should be final, so I will not speculate now on what may happen or should not happen with report that got them. Maybe later. MySQL engineers with proper permissions can set any status anyway, I've tried to explain usual and expected changes only.

I hope now you understand better why I write about some bugs on Facebook or why I care about number of "Open" bug reports at http://bugs.mysql.com. If we (community users) want our bug reports to be really taken into account, we should make sure they are reviewed and either verified or get any other final status from Oracle. Without that they will be useful only as references to similar problems for people who search or for engineers outside Oracle, but they will not influence MySQL bug fixing or development in Oracle in any clear way. Oracle customers already know what to do...

3 comments:

  1. Is this correct:
    https://raw.github.com/dveeden/mysql-history-graph/master/mysql-bug.png

    ReplyDelete
  2. Nice picture... I was too lazy to draw. Anyway, there are some corrections to do.

    "Verified" bug can not go directly to "Closed" bypassing Oracle's internal bugs database, if only bug reporter close it. But we should never do this. Once the bug is "Verified" Oracle engineers should change its state further, even if original reporter is no longer interested or see the bug in his environment.

    Bugs often go from "Need feedback" directly to "Open". Actually this happens by default every time somebody adds a comment to the bug in "Need feedback" status.

    Bug in "No Feedback" hardly ever changes its state to anything but "Open", again, after getting comment.

    I remember cases when bugs in "To be fixed later" status were changed to "Verified" (I did that few times). Also "Can't repeat" is also not a final state, ad bug may be re-opened in case of additional details about test case or testing environment added.

    ReplyDelete
  3. This is a great help for us. Thank you for the post. I have been looking for latest enhancements in My SQL and I'm glad I opened your blog site. Thank you again Sir. I will use this in my future works, in case I find any bugs that is.

    ReplyDelete