Fun with Bugs #34 - Who has fun to verify your bugs (based on 5.6.21)

This time I decided not just to review all community bugs fixed in recent MySQL 5,6 GA release, 5.6.21, but also to mention who reported them (Morgan does this recently) and, most important for this post - who verified them.

As I've explained long time ago, verification is an important part of a bugs life cycle in MySQL. We need some MySQL engineer to check the bug and make sure there is a repeatable test case for it or it is at least clear what is the real problem behind the report. Bug must be "Verified" (confirmed internally) before developers start to work on the fix.

Usually this engineer is somebody from the bugs verification team where I worked for 7 years. But we later agreed that if a bug is reported by MySQL Support Engineer or Developer, it's accepted to set it to "Verified" immediately, without waiting on somebody else from bugs verification team to pay attention and double check.

So, while reporting and fixing are, surely, important stages, verification is also important as it helps developers to concentrate on real bugs only and isolate them from most false positives and user complains. I've noted (at was surprised) that it seems more than 45% of all bugs recently fixed were actually verified by one my former colleague in Oracle since 2011 (who used to work as a regular support engineer during EMEA hours mostly), Umesh Shastry (that you may know as "Umesh Umesh" in the bugs database for the reason only him and Oracle probably knows about).

This is what we have in MySQL 5.6.21 with regards to bugs from http://bugs.mysql.com:

• Bug #70641 - 5.6 partitions use much more memory than 5.1. Reported by famous Yoshinori Matsunobu and verified by Umesh (no details shared in public). As a side note: I think it was me who introduced a habit not just to write "verified" in a comment, but also to share all the details and steps, just copy/pasting outputs from mysql command line client or terminal. (I've got this habit while following famous Oracle guru Tom Kyte.) So, I am always sad when details about the exact steps performed and versions used are no shared...
• Bug #73650 is still private and we can see no details. We can only rely on public description in the release notes: "An ALTER TABLE ... ADD FOREIGN KEY operation could cause a serious error." I know what the bug is about as I've reported a duplicate of it based on Percona Server bug report (that is public as far as I remember). many community users know all the details about the crash (let's use proper words, it was a crash). Bug is fixed. But we see nothing, neither about test case, nor about the fix (even though it's visible when one checks what changed since last version in bzr/source code repository). If you ask me, this Oracle policy is applied in a wrong way to open source software, to put it into polite words. One day I'll write long text about this approach and historical reasons for it...
• Bug #72453 - InnoDB Memcached Plugin "gets" command. Reported by qixiu xiao and verified by Umesh.
• Bug #72586 - memcached daemon crashes and restarts on second ADD operation. Reported by David Schwartz and verified by Umesh.
• Bug #72594 - InnoDB in-place ALTER failures block future ALTERs. Reported by famous code contributor, InnoDB guru and great conference speaker, Jeremy Cole. Recently we see his name in all release notes. This bug was verified by Miguel Solorzano, who was my team manager and trainer since day one in MySQL. Nice to see that he still not only manages a successful (and key!) team in Oracle MySQL Support, but also finds time to do real work.  
• Bug #72137 - inaccurate handling of srv_activity_count. Reported by famous InnoDB guru Inaam Rana and verified by Sveta Smirnova. There is no much need to explain who is Sveta and what she does - she speaks, writes and works for herself, and her work is recognized and awarded in Oracle. Check her sessions at MySQL Connect 2013, if you wonder, or read her book. I am just happy to find out that she still works on bugs processing.
• Bug #67718 - InnoDB drastically under-fills pages in certain conditions. Reported by Jeremy Cole and verified by Sinisa Milivojevic. I am happy to call Sinisa my teacher, in diplomatic skills if not in bugs processing or reading the code. If any of my former colleagues from MySQL asked what I am doing in Percona, I'd reply that I play a role of Sinisa here, "ideal" Sinisa as I wanted him to be, not the "real" one. I am not good enough in playing the role of real one, as I lack diplomatic skills, to begin with.
• Bug #71695 - Concatenation of mysqlbinlog output does not work with GTID. Reported by Sadao Hiratsuka and verified by Umesh.
• Bug #70327 - Assertion error when setting future binlog file/pos with semisync. Reported by Yoshinori Matsunobu and verified by yet another my former colleague since MySQL AB times, Mikiya Okuno. Nice to see many MySQL engineers contributing to bugs processing, not just members of bugs verification team.
• Bug #69873 - Replication stop with "Error in Xid_log_event: Commit could not be completed". Reported by Alexander Du and verified by Umesh.
• Bug #71047 - shutdown hang if semisync is enabled and slave abort. Reported by a great community contributor to MySQL, zhai weixiang. Verified by Umesh.
• Bug #72788 - HASH_SCAN seems broken: Can't find record in 't1' Error_code: 1032. Reported by Shane Bester, who probably works more on community bugs reports than all other engineers outside bugs verification team together. As I mentioned more than once, Shane is a QA department for MySQL by himself, and it had always been the case, no matter what company he worked at. Bug was verified, surprise surprise, by Umesh.
• Bug #72313 - stop sql_thread, start sql_thread causes a trx to log with a different GTID. Reported by Santosh Praneeth Banda. Not sure who verified it, but probably the reporter himself.
• Bug #72578 - Duplicate slave server_uuid in replication forum gives no specific error message. Reported by Hartmut Holzgraefe, former my colleague at MySQL AB and Sun. He helped me a lot during my first days in MySQL and he still reports a lot of bugs. Bug was verified by Umesh.
• Bug #72901 - Keep CLIENT_REMEMBER_OPTIONS for compressed connections. Bug was reported by Ramil Kalimullin, whom I miss a lot since MySQL developers conference in Riga, 2008 (as many others, it was last team meeting or conference for me till I quite from Oracle). He verified the bug himself it seems.
• Bug #73324 - Client flags are getting discarded due to incorrect assignment operator. Reported by Sujatha Sivakumar. She then verified and fixed the bug herself it seems. I am always happy to see Oracle employees and customers using public bugs database. You know why, I hope...
• Bug #72610 - Out of memory replicating using RBR and slave having different data types. Reported by Jean-Francois Gagn and verified by Umesh.
• Bug #70429 - list handling incorrect in mysql_prune_stmt_list(). Reported and verified by yet another former colleague from MySQL Support, Andrew Dalgleish.
• Bug #47641 - LIKE doesn't match when cp932_bin/sjis_bin collations are used. Reported and verified by Mikiya Okuno.
  Bug #73507 - On EL7, installation of MySQL from RPM packages could fail if postfix had previously been installed using yum. It is still private, even though I fail to see what "security implications" it may have, based on that description, comparing to any other public bug report about upgrade being screwed up. There are many, public ones...
• Bug #72066 - mysql_upgrade duplicate key error for mysql.user for 5.5.35+, 5.6.15+, 5.7.3+. Reported and verified by Jesper wisborg Krogh, again, former colleague who still works in Oracle.
• Bug #67088 - When the general_log is a socket and the reader goes away, mysql quits working. Reported by Rolf Martin-Hoster and verified by Sveta Smirnova.
• Bug #71433 - recreate+analyze OPTIMIZE TABLE T and online ALTER TABLE T may deadlock. Reported by my colleagues from Percona and great code contributor, Laurynas Biveinis. Recently he reached 100 as a number of public MySQL bugs reported. Time to review and update my list of famous bug reporters, it seems... Bug was verified by Umesh.
• Bug #72547 - Query cache not invalidated on cascade delete if db name has special symbols. Reported by Elena Stepanova, whom I met first in Riga, 2008. He contribution to QA in both MariaDB and upstream MySQL is outstanding - we can see her name in every issue of community release notes by Morgan. Bug was verified by Umesh.

So, out of 24 community bug reports 11 were surely verified by Umesh Shastry. This is almost 46% of them all. Now you know who seems to play the most important role in community bugs processing. I hope I am not the only one who noted that I am now successfully replaced in my former main job role, by a single person (whom I trained to some extent while we still worked together in Oracle).

One can probably make more conclusions based on above. For example, it looks like that most "community" bugs fixed are actually from former or current MySQL support engineers and developers, not just "users"... But let me stop on highlighting names at the moment. I am sure that some of these names were rarely mentioned in public before.