MySQL Support People - Percona Support

I planned to continue this series of posts with the one about MySQL Support engineers who joined us in Oracle while I was working there, but based on recent events in my life I'd prefer to postpone it and move directly to the team I worked for during last 3+ years, Percona Support Team.

Disclamer: In the list below I still try to pay attention to public contribution by each of engineers mentioned to MySQL Community, mostly in a form of bug reports. This is not the only way they contributed, but I have to stick to this format in current series of posts. I also base my comments mostly on my memory and quick search of public sources of information, so I may be mistaking about the roles, time periods and other details. Please, inform me if you think there is something to fix.

So, here is the list of my former colleagues who formed the best MySQL Support team in the industry in 2013-2015 while working with me in Percona:

• Miguel Angel Nieto - Miguel was my mentor during my first days in Percona, so I have to start with him. He joined Percona in 2011 and was a team manager of EMEA Support team for a long time. We always managed to work together in a productive and efficient way. Miguel is a real expert in Percona XtraDB Cluster and almost every new technology we had to support, up to MongoDB recently. He used to act as a great consultant as well when Support engineers worked together with Consulting on the same pool of issues, and he still takes SSC (Support Shift Captain) shifts, so he can be the first person from Percona whom you deal with in case of problems even today. I see 13 public bug reports from Miguel and suggest to check his Bug #77654. See also his 12 bug reports for Percona software at  https://bugs.launchpad.net/~miguelangelnieto/+reportedbugs.
• Ovais Tariq - when I joined Percona Ovais was the only Principal Support Engineer in the team. He was really great in everything technical he did, from complex data recovery consulting cases to query optimization, and on top of that he was a really good writer. It was a pleasure to read every email in issues he worked on, it was like reading a good technical blog post if not a book. I see 16 public bug reports for MySQL by Ovais. Check his replication-related Bug #70923, for example. He had also reported 20 bugs for Percona software at https://bugs.launchpad.net/~ovais-tariq/+reportedbug. In 2013 Ovais moved to Consulting and a year later, in 2014, he quit from Percona to become a Lead MySQL DBA - Automation Engineer and later a Lead Reliability Engineer at Lithium. He is also active at TwinDB site.
• Fernando Ipar - As far as I know, he was the first full time Support engineer and dedicated Support Shift Captain in Percona since September, 2009. For two years later he was a Director of Global Support. When I joined he was working in Consulting and recently he seems to do Development. But during all this time Fernando was interested in Support-related discussions and helped my colleagues promptly when needed. I see 11 bugs reported by him for MySQL, including funny optimizer regression since 5.1.x, Bug #66825. Fernando also reported 10 bugs for Percona software at https://bugs.launchpad.net/~fipar/+reportedbugs.
• Michael Rikmas (a.k.a. Mixa) - he moved to Support from Administrative team in 2011, and there he was one of the first Percona employees. He always cared about Percona operations and customers, covering endless SSC shifts any time a day, sometimes for 12+ hours in a row. He was always ready to step into anything any time a day to help, if needed. While working in Support he improved his technical skills a lot in all MySQL-related areas, so he ended up as a good problem solver and eventually quit from Percona (in 2014) to join PSCE as a Consultant. I see only one bug he reported for MySQL, Bug #62426. See also https://bugs.launchpad.net/~michael.rikmas/+reportedbugs.
• Martin Arrieta - Martin joined Percona in January, 2012, but was already a well known Support engineer when I joined. In 2013 he moved to Consulting and two years later quit from Percona in September, 2015, to join Pythian as Database Consultant. Martin had reported 6 MySQL bugs in public (he was really interested in MySQL Fabric), but he is better known as a blog posts author. I think we always cooperated really well with him. 
• Marcos Albe (a.k.a. Markus) - he is the most hard working and successful Support engineer of all times in Percona. Marcos was also my team manager after I switched to AMER Support Team. Marcos joined Percona in 2010 and by 2012 he was not only a hard working SSC, but also a full stack expert. Marcos created many useful hints for the internal knowledge base and is also know as a great speaker at Percona Live conferences (I've met him for the first time at Percona Live New York, in 2012, where he had talks on several topics). The way he works (he is ready to call you using any media, join any chat, log in promptly and spend entire night helping any customer interactively) defines famous "Percona style" of providing Support (whatever I may think about it) and gave him huge experience with all technologies even remotely related to MySQL. Marcos had reported 4 public MySQL bugs, including a 5.6.x regression one, Bug #71616, that is still "Verified". I also see 5 bugs reported by him at https://bugs.launchpad.net/~markus-albe/+reportedbugs.
• Nickolay Ihalainen - Nickolay is a classical Percona Consultant (who still works in Support for a long time), with huge real life experience of managing MySQL at scale and full application stack troubleshooting. He knows a lot about Linux, hardware used with MySQL and, as any other great consultant in Percona, is always ready to step into the issue of any complexity and length, call customer, write code, recover InnoDB data or fix a bug quickly. He works in Percona since January 2010, reported 3 MySQL bugs in public, including Bug #78051 that is still "Verified". I also see 7 bugs reported by Nickolay for Percona software at https://bugs.launchpad.net/~ihanick/+reportedbugs. I studied a lot while working with and talking to him.
• Przemyslaw Malkowski - he joined Percona in August, 2012, just before me. We were at the same "bootcamp" session in famous "Hotel Percona" and worked closely together since that time. Przemek is a Principal Support Engineer since July, 2015. He is the only one who was promoted to that level while I was working in Percona, not just joined like that. He works hard on all kinds of complex issues and is one of key experts in the industry in Galera and Percona XtraDB Cluster (He can easily parse and understand obscure, huge logs from multiple nodes, and explain the interactions and problems  that happened, quickly. This is the skill I always missed). Moreover, he is a well known bug reporter, with 22 bugs reported for upstream MySQL, including infamous Bug #78777 that mislead numerous MySQL users and customers for months, and 37 (!) bugs reported for Percona software at https://bugs.launchpad.net/~pmalkowski/+reportedbugs. Przemek is able to work with any kind of annoying customer, on issues of any complexity, and still remain helpful. I really enjoyed working with him in one team, a dream team actually!
• Nilnandan Joshi (a.k.a. Nil) - he joined Percona in May, 2012 and successfully worked there as Support Engineer till January, 2016. He managed to master all key tools and technologies in MySQL, while working hard on boring duties like SSC and bugs processing. Now Nil is a Big Data Engineer at zData Inc and is probably working with Hadoop or something of that kind. He also was visiting "Hotel Percona" with me in 2012 for a "bootcamp", where we spent a lot of time talking about the way InnoDB works (and drinking some beer). Since that time we were working in a close cooperation till his last day in Percona. Nil helped me a lot with bugs processing when I've got a task to manage it from Support side, and was one of few engineers in Support who were ready to report or properly process/verify any bug, build any Percona software, test it on all kinds of weird Linux versions and distros, using all kinds of virtual and real machines. Nil reported 13 public MySQL bugs, including Bug #79469. I also see 9 bugs reported by him for Percona software at https://bugs.launchpad.net/~nilnandan-joshi/+reportedbugs. I miss our useful work together on bugs and Percona Live sessions already...
• Jervin Real - he worked in Percona Support since early 2010 and until summer 2013, when he moved to Consulting (he worked great there as well, always ready to step in when we needed help in Support). When I joined he was a key Support engineer for APAC customers. Jervin is a well known blogger, who writes a lot about HA and other technologies for MySQL. I finally met him in real life few days ago, at FOSDEM conference, where he was speaking about TokuDB. Now Jervin is a Technical Service Manager in Percona. He had contributed a lot to MySQL Community also with his 20 public bug reports, including Bug #77715 that is still "Open". I also see 34 bugs reported by Jervin for Percona software at https://bugs.launchpad.net/~revin/+reportedbugs.
• Jaime Sicam - Jaime joined Percona Support in June, 2011 and later became a good manager of APAC team in Support. He always worked hard and helped colleagues with anything, from covering SSC shift to complex troubleshooting. He is a great engineer capable to deal with any kind of issues, but he was our key expert in anything related to PAM and authentication in general. Jaime reported 7 bugs for MySQL software, including infamous Bug #77344, and 11 bugs for Percona software at https://bugs.launchpad.net/~jssicam/+reportedbugs. We rarely were both online and working together, but I already miss him.
• Muhammad Irfan - he joined Percona Support soon after me, in December 2012. Muhammad actively started to work and cooperate with me and other senior colleagues from the beginning, and soon had become a high skilled engineer in everything related to MySQL, including XtraDB Cluster and bugs processing. He is a quite famous blogger, who was happily writing about new features, tools and best practices (while I prefer to write about bugs and problems, that is, worst practices). I am proud to get a change to help him with some posts as a reviewer. Muhammad spent fair amount of time on bugs processing and reporting as well, as we can conclude from his 8 MySQL bug reports including "Verified" Bug #73094 and 9 bugs reported for Percona software at https://bugs.launchpad.net/~muhammad-irfan/+reportedbugs.
• Fernando Laudares (a.k.a. Nando) - he joined Percona in January, 2013 and worked in AMER Support Team. I'd call him a student of Marcos, as he quickly worked out similar habits and approaches to Support, "Percona style". Fernando always tried his best to help both customers and colleagues, without any limits, and this quickly made him one of the key team members. He works hard, on many issues, day after day without becoming tired in a visible way. Fernando had written many great blog posts and is great in all kinds of automation/testing environments setups. He was fast in studying, accepting and following new technologies we were supposed to support in Percona. His community contributions also include recent enough Bug #77684 and 2 bugs reported at https://bugs.launchpad.net/~fernando-laudares/+reportedbugs.
• Jericho Rivera - with mostly sysadmin and developer background, Jericho had to work really hard since October, 2013 to reach the level of MySQL expertise we required in Percona Support. But he did that and became a key support provider in 2015, that can deal efficiently with all kinds of customer issues. He helped us to manage Support servers all this time and became a local expert in Docker and other technologies (as you can figure out from his blog posts). He had reported Bug #77073 for MySQL server (still "Verified") and 3 bugs for Percona software at https://bugs.launchpad.net/~jericho-rivera/+reportedbugs. It was really great to see how he becomes better every month, until he ended up just doing an awesome job one day.
• Peiran Song - I was really happy when she joined us in 2014. By May 2014, just in a couple of months, she already did a great job in Support, working on many complex issues. She was really good with troubleshooting, InnoDB and query optimization. We discussed not only technical details, but also procedures and values of Support, a lot. Peiran reported 6 upstream MySQL bugs, including incredible Bug #73369 that is still waiting for explanation and fix, and 8 bugs for Percona software at https://bugs.launchpad.net/~peiran-song/+reportedbugs. It was really sad to meet her in 2015 during Percona Live just to find out that she can not stay with us any more (and not because of any her faults). Peiran is a Data Architect at Smule, Inc now.
• Justin Swanhart - Justin worked for Percona at different positions (mostly as a Consultant and Instructor), since 2010. In August 2013 he joined Support as a Principal Support Engineer, and since that time we often worked together, in the same AMER team. He had got the Community Contributor of the year 2015 award, for his great contributions including (but not limited to) his software. Justin created 40 MySQL community bug reports, including Bug #76210, and I see 12 bugs reported by him for Percona software at https://bugs.launchpad.net/~greenlion/+reportedbugs. Justin worked well, if you ask me, and helped customers a lot, while being a source of unique expertise and great insights for colleagues and customers (based on his trainer and software developer experience, among other things), but one sad day in June 2015 he found himself fired in the middle of the shift. I tried to find out what was really that much wrong with his work, and found literally nothing myself, so I let others to comment on what happened and why. I think firing Justin was a very bad mistake, even though he had better NOT to comment on it in public in the way he did... I hope we'll work together one day again, maybe even soon.
• Bill Karwin - Bill worked for Percona since 2010, mostly as a Consultant and Trainer. We were really lucky that he joined Support in 2014 full time (as a Principal Support Engineer and Senior Knowledge Manager). Using his huge experience in SQL and everything MySQL, he started to play a key role in AMER team, working a lot and providing great service to customers. He is very famous for his answers at Quora and StackOverflow. He is a great writer and speaker on MySQL topics, and his book on SQL anti-patters is well known. Bill is literally second to nobody in helping MySQL Community. Check also his 9 MySQL bugs including Bug #73283 (still "Verified"), and 2 bugs at https://bugs.launchpad.net/~bill-karwin/+reportedbugs. At the end of 2014 Bill left us, probably because it was boring for him to resolve the same problems again and again for customers, instead of doing things right from the very beginning. He is now a Senior Database Architect and Software Engineer at SchoolMessenger.
• Agustin Gallego (a.k.a. Guli) - I met him first in 2012, at Percona Live New York. He worked for Percona since February, 2012, but joined Support later, in December, 2013. Since that time Agustin worked as SSC and Support Engineer in AMER Support team and played an important role in making it operational and efficient. He had reported Bug #77186 for MySQL and 6 bugs for Percona software at https://bugs.launchpad.net/~agustin-gallego/+reportedbugs. He is gaining experience every day, cooperates well with colleagues and now is a very reliable, respected and useful team member.
• Roben Paul Namuag - Paul joined Percona Support in July, 2013. In January, 2015 he left us to continue working in Percona as Remote DBA, but we still discuss technical details and, sometimes, life after that. Paul had good and harder times in Support, but he always tried to do his best and played an important role in APAC team, both as SSC and as an experienced engineer working on complex enough issues, even when there was nobody around to help. He had reported 3 MySQL bugs, including Bug #77180 that is till "Open", and 2 bugs for Percona software at https://bugs.launchpad.net/~paul-namuag/+reportedbugs. I'll remember him as a kind, helpful and friendly person.
• Akshay Suryavanshi - we chatted with with Akshay a lot before he joined Percona, he was interested in a lot of in-depth details on how MySQL works. Eventually he joined Percona Support, in March 2013, but immediately switched to Remote DBA role and had a great success there, moving later to management positions, up to DBA Team Lead and Senior position in Technical Operations recently. Then, in December 2015, he quit from Percona, to become a MySQL Engineer at Tumblr. He helped community with his blog posts and webinars, and also reported Bug #70818
• Pablo Padua - he joined Percona in 2014 as a Jr. Support Engineer, and made a lot of progress since that. Unfortunately I can not find any public MySQL bug reports by him, but he worked a lot as SSC in AMER Support Team, and he gets more experience every day. We often worked together on technical issues, when I just provided some background help, checks and comments, while he did real time work for customer. It had been a pleasure to cooperate that way, and, as a result, for many areas in MySQL Pablo now is ready to work without any assistance.
• Abdelhak Errami - he joined us as Senior Support Engineer in April 2015, when Tokutek was acquired. Since that time he played a key role in TokuDB Support, but also often helped with pure MySQL issues as well. We spent a good time together during Percona Live. Abdel often works on TokuDB bugs, check DB-922 as one of recent examples.
• Joe Laflamme - I think he also joined us some time in 2015 in the process of Tokutek acquisition. Check DB-809 for one of his TokuDB related bug reports. He coordinated TokuDB bug fixing and support during the period when I describe. Now he is a manager at Percona Technical Services.

I had not mentioned Sveta Smirnova, who joined Percona in March 2015, because she was already mentioned in the previous post. I still plan to write about managers and coordinators of Support who had never reported MySQL bugs in public separately, so stay tuned Tom Basil, Peter Farkas and Erzsebet Olsovzsky. I remember about you and I value your contribution a lot, but engineers first...

I had not named colleagues in Percona who joined after November 1, 2015 explicitly here, and this is the date of the beginning of the entirely new epoch in the history of Percona that should be described separately, if ever.