Tuesday, August 28, 2012

How to create a MySQL bug report that someone would like to read and comment on

It happens to me almost every day. I note some "bug report" at http://bugs.mysql.com that makes me think that my job is miserable... Like this, Bug #66580. What readers of such a bug report are supposed to do with it? Other than ignore?

Today I want to stay positive, so instead of cursing in public let me give some advices inspired by this great HOWTO and detailed instructions from MySQL site.

Before you send a problem report to MySQL public bugs database, please:

  • Try to find similar bugs by searching MySQL bugs database. Google search for site:bugs.mysql.com <something specific, like stack trace or message from the error log> usually works good enough
  • Try to find a solution to similar problem by searching the Web
  • Try to find a solution/reason by reading the manual (you may get links to the manual while searching the Web...)
  • Try to find a solution by reading other online documentation, forums and blogs
  • Try to find a reason for your problem by inspection or experimentation
  • Try to find a reason for your problem by asking a skilled friend (#mysql channel at irc.freenode.net could be a nice place to find some friends of this kind)
  • If you're a programmer, try to find a reason by reading or tracing the source code, inspecting core files created etc
Only if, after all these steps, you are still not able to solve your problem and still sure that the reason is the bug in MySQL, it starts to make some sense to create a new bug report.

When you send a problem report to MySQL bugs database, please:
  • Provide meaningful, specific subject/summary and description. Describe the symptoms of your problem or bug carefully and clearly
  • Write in clear, grammatical, correctly-spelled English language
  • Be precise and informative about your problem:
  1. Describe the environment in which it occurs (MySQL software version, hardware, OS, application, whatever). Provide configuration file content and attach the entire MySQL error log (compressed if it is large).
  2. Describe the research you did to try and understand the problem before you reported it
  3. Describe the diagnostic steps you took to try and pin down the problem yourself before you reported it
  4. Describe any possibly relevant recent changes in your hardware or software configuration
  5. State explicitly what exactly you consider a bug in this case, and why
  6. Ideally, you should provide a complete test case and/or instructions that any reader can use to reproduce your problem
  • Make sure the problem is repeatable with latest officially released version of MySQL software you use
  • Finally, do not report the same problem or bug more than once (unless you had found a regression bug in a recent version that was already reported before and closed). Better add your comments to existing bug report (that will be re-opened automatically in this case)
Please, remember that while it is (theoretically) possible to get free support at MySQL bugs database (and sometimes it may be even fast and good), it is neither proper nor best place to ask for it.

5 comments:

  1. We should link to this post at "Report a Bug" page.

    ReplyDelete
  2. Hi Valeriy,

    As much as I agree with everything you said, I have to note that the current routine around bugs.mysql.com makes the whole idea of spending time and effort on creating valid bug reports somewhat less appealing or even impossible.

    Here are several relatively recent examples from my personal experience.

    http://bugs.mysql.com/bug.php?id=66634 - the bug was closed as a duplicate of 66462, which would normally be the reporter's (i.e. my) fault; except that the bug 66462 is private, and it's not mine. Thus, not only could I not have googled it and avoided creating a duplicate, but also I cannot track the progress of the bug, since mine is closed, and I don't have access to the other one; so how do I know when the bug is considered fixed, and I should re-file it if it's still reproducible?

    http://bugs.mysql.com/bug.php?id=64037 - first, the bug was marked private and some time later it was just closed -- without any comment at all (none visible to me, anyway); it doesn't say whether it was fixed, and if so, in which version(s). So again, how do I know whether I should re-open it if I still encounter it?

    http://bugs.mysql.com/bug.php?id=66533 - it was initially verified, but later switched to 'Need Feedback'. I still don't know why, but it's not the point, misunderstanding does happen; the problem is, after I responded, the bug got stuck in 'Open' status. I didn't get an explanation what was wrong, so I have no idea what to do about it. Moreover, I presume as little chance as it had to ever get fixed before, it's lost now since the report is off anyone's radar.

    What I want to say is that creating meaningful test cases and bug reports is time-consuming. It's bad enough to realize that you might well be doing it in vain because it could be already known and just hidden somewhere among MySQL private bugs or in the Oracle abyss; but it's simply frustrating when at the end you don't even get to know whether the bug is fixed. So, I would not be surprised if people who did go through the trouble and got results similar to the above were disappointed enough not to try next time, thus leaving MySQL verification team to deal with those who just throw in copy-pastes of the message "mysqld restarted" with the comment "something doesn't work, please fix it".

    Regards,
    /E

    ReplyDelete
  3. Hi Elena,

    First of all, I am really sorry about all these cases and current state of bugs processing at bugs.mysql.com. It had never been my personal intention or preference to have (all) crashing bugs as private, to not mention MySQL bug numbers in changelogs, to not publish test cases or to use only Oracle internal bugs database to track real progress on bugs affecting MySQL software released under GPL (and then get outdated status for corresponding bugs at bugs.mysql.com)...

    Unfortunately today I can not explain you all the details about the bugs reports listed above, as I do not work in Oracle any more (August 31, 2012 was my last day there). Maybe Sveta will be able to help you.

    For http://bugs.mysql.com/bug.php?id=66533 we clearly see from public information that you reported it as server bug and then I verified it as server bug. Maybe I've set incorrect status as Docs. bug when transferring it to internal Oracle bugs database (I doubt, but Sveta or other current Oracle/MySQL Support employees can tell you for sure), or somebody else (Dev. lead?) had moved it to this category. I agree that having it "Open" is wrong. But, please, do not think that this bug is forgotten. Just few days passed after your last comment and after I stopped to monitor bugs database every day. There are other people at MySQL Support (Sveta, Miguel, maybe others will step in to replace me) who should regularly review all open bugs. Problem will be fixed eventually, trust me. If not, we will blog about it and force changes this way.

    I know how much time it often takes to understand the problem behind the bug and create a simple repeatable test case. Your test cases had always been just great. Please, do not let current problematic policies or deficiencies in processing of some of the bugs to stop you from contributing test cases to http://bugs.mysql.com.

    I still think that this is the best place for generic MySQL bug reports. Bug reports at Launchpad for specific branches or in some company-specific knowledge bases are not a replacement. At least while Oracle still maintains MySQL server as free and open source software and contributes big share of the bug fixes fox this code.

    Thank you for your hard work on MySQL QA over these years at Sun, Oracle and then Monty Program AB.

    ReplyDelete
  4. @elenst:

    Hi Elena,

    I can not comment everything, but still.

    > http://bugs.mysql.com/bug.php?id=66634 - the bug was closed as a duplicate of 66462, which would normally be the reporter's (i.e. my) fault; except that the bug 66462 is private, and it's not mine.

    When we close bugs as duplicate we do it to track all activities on them in single place. We don't call it your or somebody else fault. This is just done to keep developers job simpler.

    > http://bugs.mysql.com/bug.php?id=66533 - it was initially verified, but later switched to 'Need Feedback'.

    This is because Paul wanted feedback. I fixed it now.

    ReplyDelete
  5. Hi Valeriy, Sveta,

    I know you can't comment on the privacy regulations and that it's certainly not the verification team's fault. I just think that it's an important point to consider if you are really wondering why there are so few good external bug reports nowadays. The lack of proper feedback is perceived as the lack of respect.

    >> I still think that this is the best place for generic MySQL bug reports

    I agree, that's why we report generic MySQL bugs to bugs.mysql.com whenever we have a decent test case working on MySQL versions.

    >> August 31, 2012 was my last day there

    Ah, that's why the "former" Entomologist... I thought you were still in doubt, as it came with a question mark. So, congratulations or condolences, whichever is more appropriate :)

    ReplyDelete