Thursday, July 18, 2013

Fun with Bugs #17 - Oldies but Goldies

I've just noted the date... 8 years ago I had my first official working day as a support engineer in bugs verification team of MySQL Support at MySQL AB. Why not to celebrate this anniversary with a blog post about bugs?

So, here they are, 12 oldest bugs in MySQL software that are still just "Verified" (it should mean they are accepted, but not yet fixed):
  1. Bug #2 - MySQL Connector/J doesn't make toast. I knew that Connector/J must be the most broken MySQL software (as I hate Hibernate). Now you can see how much it is broken, and nobody cares to fix it since 2002! This is a real shame...
  2. Bug #199 - Innodb autoincrement counter is lost on restart. This great report from Peter Zaitsev is still "Verified", since 2003. It became a documented "feature" in the process, but still "Verified" status means that over these years we all agree - it's broken by design! Let's hope MySQL 5.7 will fix it.
  3. Bug #363  - SHOW SESSION VARIABLES LIKE 'x' != SELECT @@SESSION.x. I spent some time verifying the original test case, and it seems in 5.6.12 it is no longer repeatable (but it seems more general things were discussed there:

    mysql> select version();
    +-----------+
    | version() |
    +-----------+
    | 5.6.12    |
    +-----------+
    1 row in set (0.01 sec)

    mysql> show session variables like 'sql_log_bin';
    +---------------+-------+
    | Variable_name | Value |
    +---------------+-------+
    | sql_log_bin   | ON    |
    +---------------+-------+
    1 row in set (0.15 sec)

    mysql> select @@session.sql_log_bin;
    +-----------------------+
    | @@session.sql_log_bin |
    +-----------------------+
    |                     1 |
    +-----------------------+
    1 row in set (0.00 sec)
  4. Bug #1382 (and it's later duplicate, Bug #23385, reported by my friend Tonci) - Server returns only VARCHAR as type for any metadata sent to the client. Metadata server returns had always been a problem for all MySQL connectors. Still "Verified". Now you have "Affects Me" button at least...
  5. Bug #1396 - "Lost Data with delayed insert / mysqlhotcopy or lock and flush". It was declared a feature request to implement a workaround to make this more safe. Do not use INSERT DELAYED and/or mysqlhotcopy, what else?
  6. Bug #1939 - "Wrong case sensitivity for table names when in "ANSI" mode". We all know setting lower_case_table_names=1 is the ultimate solution, and hardly this is going to change while we still have .frm files, but the bug is still "Verified".
  7. Bug #1956 - "Parser allows naming of PRIMARY KEY". Still happens with 5.6.12, and surely in SHOW CREATE TABLE you will not see the name. "One day at a time, Penny, one day at a time".
  8. Bug #2122  - "changing hostname confuses master or slave". We all know since day one with MySQL replication that binary logs should get explicit names. Let me quote one comment form the bug report though:

    [13 Sep 2007 11:54] Lars Thalmann
    We are analyzing this and will fix it after 5.1 is released. It is, unfortunately, not likely that this will be fixed in 5.1, but we prioritize it for the next version.

    Still just "Verified".
  9. Bug #2261 - "Can't use @user_variable as FETCH target in stored procedure". So, what? Just scroll down and read last comment from the best MySQL architect of all times, famous writer and bug reporter of the past, Peter Gulutzan. I was drinking wine with him near the hotel entrance last time we met (2008 AFAIR)!
  10. Bug #2742 - "fields-escaped-by only allows single character separator!". Not sure what is the status and the real problem, but look at the bug reporter's name: "Are you mortal Then prepare to die." It's the last still active bug report from this famous personality who was well known during good old times of MySQL AB.
  11. Bug #2812. When executing SHOW VARIABLES in a prepared statement stmt->field_count is 0 instead of 2. Still "Verified" and you know what these mean in the bug report: "Triaged: D2 (Serious) / R4 (High) / E4 (High)". It means that bug was considered serious, but efforts to fix and risk to introduce regressions or other problems was estimated as high. Poor MySQL connectors' developers, really...
  12. Bug #3052 - "Rollback in stored procedure doesn't close cursor". Now read the final comment:

    [22 Aug 2006 15:02] Konstantin Osipov
    This is too big to be fixed in 5.0, will be fixed in 5.1

    You know what, it's still in 5.6.12:

    mysql> select version();
    +-----------+
    | version() |
    +-----------+
    | 5.6.12    |
    +-----------+
    1 row in set (0.00 sec)

    mysql> delimiter //
    mysql> create table t (s1 int) engine=InnoDB//
    Query OK, 0 rows affected (0.73 sec)

    mysql> insert into t values (55)//
    Query OK, 1 row affected (0.06 sec)

    mysql> create procedure p2 () begin
        ->   declare v int;
        ->   declare x cursor for select s1 from t;
        ->   open x;
        ->   rollback;
        ->   fetch x into v;
        ->   set @x = v;
        ->   end;//
    Query OK, 0 rows affected (0.24 sec)

    mysql> call p2()//
    Query OK, 0 rows affected (0.07 sec)
I can continue like that for days and 20 more posts... Reading (and processing) MySQL bug reports is a real fun! That's why I do it (with small breaks) for 8 years already and not going to stop. It's great reading no matter what are looking for - technical insights, workarounds, patches, hints, nice stories or just to something to laugh at.

Saturday, July 13, 2013

Fun with Bugs #16 - read the fine MySQL 5.6 manual...

MySQL 5.6 has many new features and some of them change usual DBA procedures and formerly well known facts/limitations. It's important to have all these properly documented now, when MySQL 5.6 is GA for 6 months already and is supposed to be widely used in production. So, I think it's time to check what documentation problems still remain.

Here is the list of active bug reports in Server: Docs category for version 5.6, starting with recently reported:

  • Bug #69717 - "DML statements replicated via RBR are NOT logged in the general query log ". I had to report this while working on customer issue and trying to explain why there are no DML statements in slave's general query log. This feature was new in 5.1 and even somewhat documented at http://dev.mysql.com/doc/refman/5.6/en/replication-sbr-rbr.html, but I'd prefer to see this explained at http://dev.mysql.com/doc/refman/5.6/en/query-log.html also. Moreover, in 5.6 we may be able to log somethings useful, as my colleague Justin Swanhart noted:
    On 5.6 perhaps the general log should log the query event associated with the RBR if it is available?
    So, it seems this is not only about the manual, but also about re-opening good old feature request, Bug #14722 for 5.6+.
  • Bug #69711 - "Explain what operations lead to specific "Waiting for... lock" states", is for 5.5+ actually, where metadata locks were introduced, but it makes sense to mention it in context of 5.6, as I do not want to see major 5.6 new features not entirely documented 2 years from now. This may happen, as the bug report proves, if we'll just keep silence. Use "Affects Me" button at least, please.
  • Bug #69697 - "Manual has not enough details on how to use transportable tablespaces" is a good example of what was missing in 5.6 manual on one of cool new features. It happened so that actual current content of that page, with detailed explanations, was pushed the day before I reported the bug, but was not yet visible. Anyway, explanation of  messages in the error log in the process of "importing" InnoDB tablespace is still missing. Also note that many documentation requests are satisfied almost immediately these days. Check my Bug #69701 for a great example of how it works (and what problems you may get while using transportable tablespace feature). So, keep reporting bugs, even if this is just about the missing phrase in the manual, and let Oracle engineers do their part of the job!
  •  If you upgrade to 5.6 after using 5.5 for a long time on Linux, note that mysql_install_db does NOT read /etc/my.cnf any more. Check Bug #68807 and Bug #69677 for the details and some consequences and workaround. Again, nice to see Oracle MySQL engineers still using public bug database, as in this case. I think for documentation requests it's a must - community should really know what is NOT properly documented yet. This hardly can lead to any security problems...
  • Bug #69630 - "MYSQL_FIELD structure has incorrect information". Even though the bug is still "Open" and may end up without any fix even in the manual, I still think it's a useful reading for people who use MYSQL_FIELD.charsetnr content in their code. Set your expectations according to the reality, please.
  • Bug #69490 - "Memory usage of P_S". I hope you already know that PERFORMANCE_SCHEMA is enables in MySQL 5.6.x by default and that you will have to pay for this with performance decrease (in some cases) and additional memory used (in all cases, like 170K per connection in max_connections, see details in the bug). I'd like to see the details shared by Marc Alff in Bug #68514 clearly mentioned in the manual. I hope I am not alone...
  • Bug #69199 - "STATS_SAMPLE_PAGES not documented in "CREATE TABLE" page". It's clear what is missing, so let's hope it will be fixed soon. Nice new feature that users may not be even aware about.
  • Bug #69135 - it seems that manual should still become more clear on what exactly one should do in MySQL 5.6 to make replication really crash safe.
  • Bug #68097 - "Manual does not explain that some P_S instruments must be enabled at startup". I've reported this almost 7 months ago, when 5.6.9 was released. But now at least there is some work in progress it seems. They say it is not trivial to document...
  • Bug #67023 - "EXPLAIN FORMAT=json output not documented". It would be really nice to see side-by-pside comparison of traditional output and output in JSON format. For me it is still not clear if (or what) additional information one may get from JSON format (and I tried to figure out with simple examples, really). As you can see some MySQL 5.6 documentation requests are not lucky and may hang around in "Verified" state for many months...
  • Bug #66397 - "Docs info on installing on Windows is not updated". Documentation now clearly describes that you get MySQL Installer from dev.mysql.com while downloading .msi, but I do not see information about two kinds of installers (check http://dev.mysql.com/downloads/installer/5.6.html) or that one may find good old .msi packages on FTP mirrors (not sure, had not checked for a long time). Time to add missing details and close this bug maybe?
  • The last but not the least, tow documentation requests related to new MRR optimizer feature are still just "Verified", since 2011! Check Bug #62676 and Bug #60864. Add to this the fact that default setting mrr=on led to one of few performance regressions in MySQL 5.6 (I had written about it here) and you'll understand my request to have it disabled by default for upcoming Percona Server 5.6 GA better (not that it's going to be satisfied, who knows...)
That's all, folks. Based on number of bug reports, MySQL 5.6 manual seems to be in good enough shape. Do you agree?

Sunday, July 7, 2013

Fun with Bugs #15 - Recent News and Hawthorne Effect Studies

Let me present a quick review of new and recently verified MySQL bug reports (mostly in 5.6.12, but not only). Surely I have to start with this request that many my colleagues had already mentioned in their blogs:

Bug #69558 - Put *all* know bugs into the public bug tracking system at bugs.mysql.com. We may argue on how and when this should apply to "security" bugs, but automatic bi-directional replication (even if delayed) with Oracle internals bugs database is what I was also asking for since we were forced to use it. Click on "Affects Me" button there and let's hope that some day Oracle will publish list of bugs that affect most of community users and may even try to take this into account while making decisions.

I have good news for everybody who was following MySQL 5.6 Performance Schema overhead discussions. Bug #68413 is now "Verified". So it is officially confirmed now that there are use cases when performance overhead from Performance Schema being ON with default settings is at least 10%. We all knew that, and there was work in progress to reduce overhead for months already, but for some reason it took a long time and notable efforts from several people (starting with original bug reporter, surely) to get the bug formally "Verified" instead of being formally ignored.

Now, let me remind Oracle engineers about some bugs that probably are fixed already, but still are NOT "Closed" in our public bugs database:
  • Bug #68022 - "DROP TABLE slow when it decompress compressed-only pages". As far as I understand it should be fixed in MySQL 5.5 at least, but it still hangs around "Verified".
  • Bug #56332  - "Performance regression in DROP TABLE performance post 5.0.77". It is probably "Won't fix" for 5.0.x that is no longer supported, was fixed in 5.5 long time ago, and (check Bug #69316) should be finally fixed (again) in upcoming MySQL 5.6.13 and 5.7.2.

Unfortunately we still see regressions in new releases from Oracle. One that everybody who uses more than one datafile for shared InnoDB tablespace should know about is Bug #69623 - "since 5.5.32 & 5.6.12, innodb cant start with own multi-file tablespace". See Bug #69573 also. I already had to deal with customer affected (that upgraded to Oracle's 5.5.32), so the problem for production environments is real.

Bug #69653, "Use of uninitialized pthread_getspecific() key in debug builds", may sound unimportant to you, but it is yet another evidence that there are things to improve in code review and analysis in Oracle.

Good old Bug #62255 ("DROP USER can't drop users with legacy upper case host name anymore") had got yet another duplicate this week. Better just name your hosts all lowercase to avoid problems like this.

Should I even remind you that it hardly makes sense to use ORDER BY while defining views if you do not add LIMIT clause at the same time? If I should, please, check Bug #69678 - it shows that you may get hit of a regression bug in this case. One more change in behavior in 5.6.12 vs 5.5, but IMHO it's just easier to avoid this kind of views.

Now some "personal" things to mention. I keep wondering is there any work in progress on Bug #68487, "MDL hash can still be concurrency bottleneck"? I am also going to abuse this blog post to remind about and old documentation request I've made, Bug #68097. I had not double checked, but are we sure that all details on enabling PERFORMANCE_SCHEMA instruments are now properly documented? Hardly to assume this, as bug is still just "Verified". I also have one bug to add to the list of replication problems in 5.6.12, Bug #69574 - Slave crashes when applying row-based binlog entries in cascading replication...

Finally, some background theory. While trying to understand if my MySQL bugs-related activity can be useful at least theoretically I started to read about observer's paradox and ended up on a page describing Hawthorne effect:
"The Hawthorne effect is a form of reactivity whereby subjects improve or modify an aspect of their behavior being experimentally measured simply in response to the fact that they know they are being studied"
So, I do study community bugs processing in Oracle and describe my findings in public, and as a result engineers involved may start to improve bugs processing, no matter what I actually noted during my study or report in my Facebook and blog posts (that many consider annoying). Just because they know that they are being studied... Sounds like a plan! Stay tuned.

Friday, June 28, 2013

Fun with Bugs #14 - InnoDB in MySQL 5.6

InnoDB improvements in MySQL 5.6 are well known. One of the key reasons to upgrade to MySQL 5.6 for most users is to get the benefits of improved performance, scalability, new monitoring features and fulltext indexes support in InnoDB.

Is there anything to double check before assuming that InnoDB in MySQL 5.6 is just better than any older version for any practical purposes? Let's review known public InnoDB-specific bug reports. Here is my "Top 10" list, as of MySQL 5.6.12, starting with most recent reports:

  1. Bug #69424  - maybe I miss something (I am not the only one though), but I see no way to continue using raw devices (on Linux at least) to store InnoDB data. You had working raw device in 5.5.32, then you upgrade to 5.6.12 and just can not start MySQL any more. Check this bug for the details and maybe you'll find out what we all miss in this case.
  2. Bug #69356 - insert (or change, if you prefer) buffer is a known source of problems for InnoDB, historically. This is one more case when server can not start up successfully because "ibuf_restore_pos failed when trying to restore cursor to the first record of a leaf page". There is work in progress on similar/related internal bug report it seems, so we may expect some fix soon.
  3. Bug #69325 - with default settings MySQL may consume unexpected amounts (too much) of memory while altering InnoDB table with many partitions. Take care!
  4. Bug #69316 - Drop/Alter table takes much longer time in 5.6 than 5.5 (with big InnoDB buffer pool that is mostly filled). We all remember long story of fixing similar cases in MySQL 5.5, so here we have really bad news: it seems some of these fixed had not made their way to MySQL 5.6, or there is a regression for some other reason. But that's it - things you expect to work fast enough for a long time already may be slow again in MySQL 5.6.
  5. Bug #69236 - skip the beginning of the bug report (impact of utf8 or client problems is a topic for other reports and posts) and move on to perf results at the end. This is what was verified, I assume. MySQL 5.6 is spending a lot more time in rec_get_offsets_func, trx_undo_report_row_operation and btr_cur_optimistic_insert functions comparing to older versions. This is one of the reasons for performance regressions for single-threaded workloads reported by Facebook. Note that on slave it is usually single SQL thread that does a lot of work, so this regression matters.
  6. Bug #69179 - persistent statistics for InnoDB tables in MySQL 5.6 is not really persistent. Even with read only workload statistics changes when you query from information_schema.partitions, and this may cause query plans to change.
  7. Bug #69174  - it seems page cleaner thread may sleep too long time (is not working hard enough) in some cases. Hardly to be fixed any time soon. Bug #69170 has somewhat similar future it seems in terms of getting fix from Oracle. Some day, maybe.
  8. Bug #69168 - not sure if this is in 5.6 only, but SELECT COUNT(*)...GROUP BY sometimes returns wrong results on partitioned InnoDB tables. Check this to make sure your queries are not affected.
  9. Bug #69141 - SELECT from InnoDB table hangs being run in parallel with replicating LOAD DATA. It seems you may get a surprise after upgrading slave to MySQL 5.6...
  10. Bug #69002 - interesting finding by Domas: InnoDB reads transaction logs on writes. Let me quote: "With large transaction logs that means that either InnoDB will hit this a lot, or amount of memory equal to innodb transaction log will be wasted". If you are a big fan of Domas and/or Sinisa and/or discussion it's worth reading, no matter how big are your logs or what MySQL version you use.
I had to stop now because of "Top 10" format (too many bugs to list before my Bug #68079 that questions scalability improvements in a simple enough real life use case anyway, so why bother...). I've skipped numerous bug reports about "wasted work",  or "redundant code" - while important for further performance improvements, they are hardly important for those who consider upgrade to MySQL 5.6. Something that looks like a regression or change in behavior, or no real improvement in performance, or lack of documentation is more important in this case, I think. I've also skipped Windows-specific (legacy) problems like Bug #69326 and bugs that are surely repeatable with older MySQL server versions as well. So, it's as much about MySQL 5.6-specific surprises as possible.

To summarize: you may get problems at InnoDB level as well when upgrading from older versions to MySQL 5.6. From inability to use raw devices to slow DROP table (again), wrong results of your queries and huge memory use/swapping. Or you may just enjoy improved scalability and great new features... It depends, as usual. Just make sure you reviewed know bug reports before making final decision on upgrade and set your expectations accordingly.

Monday, June 24, 2013

Fun with Bugs #13 - MySQL replication and two-way communication

I hope you had noted this already, but in case you missed it, please, read this post by Matt Lord and check any bug at http://bugs.mysql.com. As soon as you log in to your Oracle account, you can vote for bugs and feature requests! I hope that eventually somebody will publish lists of "Top N Most Wanted" fixes based on number of users who clicked on this great "Affects Me" button.

If you plan to use this new feature to express your needs while given a chance, why not to start with replication-related bugs in latest and greatest MySQL 5.6.12? Here is my "Top 10" list (starting with recently reported):
  1. Bug #69444 - just do not assume that replication in MySQL 5.6 is magically crash safe in all cases. DDL and MTS (multi-threaded slave, in this context) may still make starting replication from proper position a problem sometimes.
  2. Bug #69369 - when GTIDs and MTS are used slave's SQL thread just stops when binlog rotation happens on slave (related to binlog group commit it seems).
  3. Bug #69341 - semi-sync replication is slow when changes are done by many clients on master. There is a preliminary patch in the bug report, so you may want to check it (no public feedback from original bug reporter so far).
  4. Bug #69135 - probably just a documentation issue formally, but still: don't forget to add sync_master_info=1 when master_info_repository = TABLE and maybe more if you want replication to be really crash safe in 5.6. Check comments in the bug report carefully, please.
  5. Bug #69097 - mysqld scans all binary logs on crash recovery. This is really serious and may be considered a performance regression of a kind. I am surprised that there are no public comments since April 30 and I really hope 5.6.13 is going to fix this bug.
  6. Bug #69096 - GTID_NEXT_LIST session variable is not visible. As a result, there is no way to recover from Bug #69045, so make sure you use MySQL 5.6.12, not any older version.
  7. Bug #69095 - replication in 5.6 (including 5.6.12) may break with GTIDs enabled and master changes from SBR to RBR. Bug is still "Open" and similar problem could happen with 5.5.31 it seems, but still make sure you review this case if you plan to switch to GTIDs in 5.6.
  8. Bug #69059 - for many real life use cases it's not possible to turn down and restart the entire database topology simultaneously in order to enable GTIDs. So, how to start using this feature in production, while you upgrade from older 5.x.y to 5.6? Still no public answer since April 24 to this question from Facebook, unfortunately.
  9. Bug #68953  - some of binlog write errors, namely originating in MYSQL_BIN_LOG::do_write_cache, are silently ignored. This may be considered as a regression comparing to 5.5 (as new code is affected) and, in any case, is not any good.
  10. Bug #68892 - Invalid use of GRANT command breaks replication. Surely DBA can break it using some other way, but writing to the binary log something that just can not be executed is wrong in any case, I think.
Let's stop at this stage. The goal of this issue was not to provide a complete list of replication bugs in 5.6.12, but rather to give a yet another partial answer to the question on what users upgrading to MySQL 5.6 should care about. To put it simple: do not assume that concurrency improvement, MTS, GTIDs, crash-safe replication and other new replication features just work well together by default.

Previous partial answer was given here. Few more issues on InnoDB, installation or upgrade/downgrade problems and PERFORMANCE_SCHEMA are still needed to get the entire picture...

Sunday, June 23, 2013

Fun with Bugs #12 - MySQL Cluster 7.3 GA

I had always tried to avoid all kinds of clusters, from Oracle RAC to MySQL NDB Cluster and Percona XtraDB Cluster, as much as possible. But these days clusters become common and it seems new developments in this area can not be just ignored. So, I decided to devote this issue of "Fun with Bugs" to MySQL Cluster 7.3, that was released as GA this week and still is in the news.

The release was mostly about adding foreign keys support (one of the features that some users were missing for years comparing to InnoDB and other cluster database solutions). At the same time, MySQL Cluster is now based on MySQL Server 5.6 code. I've decided to quickly check how community adopted 7.3 and what it means in terms of bug reports.

If one would just search for active bugs in version "7.3" at the public bugs database using its own advanced search form, she would find only 8 bugs, of them only the following are recent and/or seems serious:
  • Bug #69528 - ORDER BY with JOIN may produce wrong results in 7.3.2. This is probably related to one of optimizer regression bugs in MySQL 5.6 (check previous issue for some of them) and is not specific to NDB storage engine, but still may become a regression that one can easily hit.
  • Bug #69510 - LIKE just does not work as expected with NDB tables. This is really weird, if you ask me, to see this bug in new GA release. Read that report to check what others think... It seems a well known old problem, probably with a known solution, that somehow ends up not fixed in too many MySL Cluster releases. When I ask what's going on with MySQL QA these days it's exactly this kind of bugs that makes me worry about it, no matter how much Oracle really invests into the process. It seems that some procedures are just not there or still not followed.
  • One of the new features of MySQL Cluster 7.3 is "NoSQL JavaScript Connector for Node.js". Unfortunately it is not bug free, check Bug #69509. Good that it's al;ready work in progress and was reported by Oracle MySQL engineer in public bugs database. It shows that engineers care, still, if not to prevent the bug then to inform about it as soon as it is found.
Other bug reports are either old enough or not of wide importance. There are minor problems with installer on Windows (check Bug #69120 and Bug #69112 from Shane), ndbd does not report failure status when it runs as Windows service (check Bug #69063). Problems with table names case (non-) sensitivity on Windows, that used to hit InnoDB, seems to be reported many months ago for new MySQL Cluster foreign key feature (check Bug #67354), and it is still not clear if anybody is going to do anything about it. That's all "Verified" bugs for now in 7.3.

Does it mean that MySQL Cluster 7.3 GA is nearly bug free and safe to upgrade to? Unfortunately no, and the reason (besides 3 bugs above) is simple: it is based in MySQL 5.6, moreover, 5.6.11 at the moment, so you should expect to see dozens of bugs that are still not fixed in MySQL 5.6. So, it's time to test this release for sure, and it's time to check active bugs in MySQL 5.6, but, IMHO, hardly it's time to think that upgrade to MySQL Cluster 7.3 may be seamless and not prune to regressions.

Saturday, June 22, 2013

Fun with Bugs #11 - Top 10 Optimizer Regression Bugs in MySQL 5.6

I've got a question from colleague last night on what bugs should users take into account if they plan to upgrade to MySQL 5.6 now. Simple answer is: it depends. If one of the new features or scalability improvements are really important, then bugs in other features or clearly identified problematic use cases may be just ignored or avoided.

But to be on a safe side users should at least check if they are (or may be) affected by a known regression bugs, when new version is slower or produce wrong results or crashes in cases that worked without problems before.

List of bugs in MySQL 5.6 that can be formally considered as regressions comparing to previous major versions would be long enough for a single post. So I'd like to concentrate on regression bugs in Optimizer here:
  • Bug #69471 - "UNION of derived tables returns wrong results with "1=0/false"-clauses". Even current code of 5.6.13 is affected. Unfortunately all kinds of programs that generate SQL based on user input often add these "1=0" or "1=1" clauses to the list of conditions, and MyDQL 5.6 may start producing wrong results for queries with default settings.
  • Bug #69410 - it's not the first time when queries with LIMIT clause perform actually worse that queries without LIMIT, especially for InnoDB tables with many indexes defined and ORDER BY clause in the query. Here index condition pushdown optimization comes into play and leads to slower execution comparing to 5.5.x. Work in progress already, so maybe MySQL 5.6.13 will fix this.
  • Bug #68979 (with recent Bug #69390 considered a duplicate by Miguel but no way for us to check or confirm as user's schema is private). New feature, "Delayed materialization of derived tables", may lead to different plan comparing to 5.5.x and performance regressions. Looks like users that use derived tables a lot in their SQL statements should be very careful while testing with MySQL 5.6.
  • Bug #69350 - Shane had found that stored procedures doing things as simple as select 1 into `j`; in a loop perform notably worse in MySQL 5.6 comparing to MySQL 5.5. I am not sure if "Optimizer" is a proper category for this bug, but still this is something to take into account if this use case is typical for your application.
  • Bug #69268 - another public bug report from Oracle engineer. It seems that simple query like SELECT * FROM a LEFT JOIN b ON a.id = b.id GROUP BY a.id; may start to return different results in 5.6 comparing to 5.5. One may speculate on how correct is it to rely on extended GROUP BY feature of MySQL, or can this be considered an extended GROUP BY if the right table has only one row, but changes in results for simple queries are always surprising...
  • Bug #69233 - this bug is probably related to fraction of seconds now supported for date/time in 5.6 (this was a problem for Percona data recovery tools as well, by the way) and may affect ODBC-based applications. Anyway, inconsistent results of queries are never good.
  • Bug #69219 - I am happy to see Oracle MySQL engineers still reporting bugs at public bugs database, but hardly users are happy when statement like CREATE TEMPORARY TABLE ... SELECT is 4+ times slower on 5.6.12 than on their good old 5.1.
  •  Bug #69005 - check this if you use ORDER BY LOWER(column) in any query. It may produce wrong results in MySQL 5.6.x easily. As simple as that, and still not fixed.  
  •  Bug #68919 - yet another great case when new optimizer feature (DS-MRR in this case) leads to performance regression, again LIMIT involved. Fortunately you can just disable this feature with optimizer_switch="mrr=off";
  •  Bug #68897 - one more case of wrong results in MySQL 5.6 when user variables, derived tables, GROUP BY and ORDER BY are involved.
I think we can stop now, to be able to add nice "Top 10" clause to the title.

To summarize, if you plan to upgrade to 5.6 now and use derived tables, user variables, GROUP BY, ORDER BY or LIMIT clauses in your queries, please, review bug reports above, other active optimizer bugs affecting 5.6 (I hope you know how to find them) and text your applications carefully. You should expect both performance regressions and wrong results from MySQL 5.6.12 in these cases, and not all of them are easy to workaround. You may have to disable new optimizations or rewire your queries.