We are quickly approaching opening up the MySQL Query Analyzer for general beta and I wanted to pass along an open invite to the following related and informational events.
On 8/13, I will be doing a micro level presentation on MySQL Enterprise. Please attend and learn more about the database software, monitoring and advisor services and support solutions that make up a subscription. I plan to do a demo of the Enterprise Monitor and the new Query Analyzer; that alone makes attending worth the price of admission (in this case 45 minutes of your time!). Learn more and register here.
On 8/20, I will be doing a presentation on the new Query Analyzer. This will be a technical discussion around how DBAs monitor for bad queries now and how the Query Analyzer makes the job much easier. This will be a good time to learn about getting in on the beta too. Learn more and register here.
Thursday, July 31, 2008
Tuesday, July 29, 2008
MySQL Enterprise Monitor: The Secret about Agents
I am often asked why we chose a distributed, web-based architecture for the Enterprise Monitor application. For those not familiar with how the app is deployed, the Enterprise Monitor is typically installed within a customer firewall and is comprised of 3 components:
- a lightweight agent written in C, that is deployed to each monitored server to collect MySQL, OS data and Query content and diagnostics
- a centralized server that monitors the collected data and queries, sends out alerts and serves up the supporting web application
- a MySQL repository that holds the collected data and queries
A typical deployment looks something like this:
Having come to MySQL from Embarcadero Technologies, I understand that many customers are somewhat uneasy with loading an agent on their production database servers. In fact for environments with a small number of developers, applications, and databases I actually prefer a client-centric solution. With this in mind I use the following guidelines when deciding if a new administration/monitoring product or app should be client or web-based (please note that these do NOT apply to development or design tools, which are best rendered in client side apps):
Local, client-based apps are best for:
- Small development shops (3 or less developers). I want the available connections to my production servers to be paying customers not DBAs or Developers!
- Small applications with a small number of MySQL servers. Most client side apps provide single server connections and visibility making navigation a bit kludgy. Some ISVs have the multi-server view thing figured out, but most do not.
- Departmental applications with limited end users. The SLAs tend to be less strict and network bandwidth issues around pulling data to/from dedicated client connections is less of an issue.
- Small, localized corporate campuses. Typically client based solutions require the footprint of the app to reside on a specific desktop, device, etc. This requires the user to be somewhere close to an enabled workstation to well, uh, use the app.
Distributed, web-base apps are best for:
- Large shops with many DBAs, Developers, Sys Admins. In most web-based deployments all app users can be serviced by a single connection and data collected by an agent that is running on the monitored data source. Minimal connections, maximum efficiency.
- Monitoring and managing a large number of backend MySQL servers. This is made easy because typically an agent based architecture scales better than one that relies on adding more clients (and connections) to an environment. Most deployed agents serve as "daemons" that keep track of things MySQL up/down status, connection status, etc, all without the user having to initiate a connection.
- Monitoring MySQL from anywhere. Basically all you need is access to a browser. Enough said.
So, why did we choose a web-based architecture for the Enterprise Monitor? Well, it seems most of our users are split across these guidelines; most have small, nimble MySQL DBA and Dev teams, but an ever growing number of MySQL servers. We opted to provide a tool that would help them scale their resources to monitor/manage more MySQL servers with less time and effort.
- a lightweight agent written in C, that is deployed to each monitored server to collect MySQL, OS data and Query content and diagnostics
- a centralized server that monitors the collected data and queries, sends out alerts and serves up the supporting web application
- a MySQL repository that holds the collected data and queries
A typical deployment looks something like this:
Having come to MySQL from Embarcadero Technologies, I understand that many customers are somewhat uneasy with loading an agent on their production database servers. In fact for environments with a small number of developers, applications, and databases I actually prefer a client-centric solution. With this in mind I use the following guidelines when deciding if a new administration/monitoring product or app should be client or web-based (please note that these do NOT apply to development or design tools, which are best rendered in client side apps):
Local, client-based apps are best for:
- Small development shops (3 or less developers). I want the available connections to my production servers to be paying customers not DBAs or Developers!
- Small applications with a small number of MySQL servers. Most client side apps provide single server connections and visibility making navigation a bit kludgy. Some ISVs have the multi-server view thing figured out, but most do not.
- Departmental applications with limited end users. The SLAs tend to be less strict and network bandwidth issues around pulling data to/from dedicated client connections is less of an issue.
- Small, localized corporate campuses. Typically client based solutions require the footprint of the app to reside on a specific desktop, device, etc. This requires the user to be somewhere close to an enabled workstation to well, uh, use the app.
Distributed, web-base apps are best for:
- Large shops with many DBAs, Developers, Sys Admins. In most web-based deployments all app users can be serviced by a single connection and data collected by an agent that is running on the monitored data source. Minimal connections, maximum efficiency.
- Monitoring and managing a large number of backend MySQL servers. This is made easy because typically an agent based architecture scales better than one that relies on adding more clients (and connections) to an environment. Most deployed agents serve as "daemons" that keep track of things MySQL up/down status, connection status, etc, all without the user having to initiate a connection.
- Monitoring MySQL from anywhere. Basically all you need is access to a browser. Enough said.
So, why did we choose a web-based architecture for the Enterprise Monitor? Well, it seems most of our users are split across these guidelines; most have small, nimble MySQL DBA and Dev teams, but an ever growing number of MySQL servers. We opted to provide a tool that would help them scale their resources to monitor/manage more MySQL servers with less time and effort.
Monday, July 28, 2008
MySQL Enterprise Monitor: Competition is a good thing!
As the Product Manager for MySQL Enterprise and the Enterprise Monitor I am constantly being asked questions from our Sales team, prospects, customers, etc. about how our products stack up against competing products. This is tough for a PM because competitive situations change with each new release cycle and ISVs (both free/open and commercial) with agile development practices can deliver new features in very short order. Further, getting into a feature-feature discussion is a no win situation because someone will ALWAYS have more check marks. Also, I tend to be more positive about competing products because a) healthy competition makes us all better and b) my competitors enable more people to use MySQL to build apps that will most likely need MySQL support and c) the best support for MySQL comes under a MySQL Enterprise subscription! With those things in mind you will *never* hear me or the Engineering team I work with bad mouth or otherwise run down a competing MySQL monitoring product.
When asked about competing MySQL Monitoring products I think it is best to focus most on the value the Enterprise Monitor provides by asking some simple questions:
- Does product X allow you to see and monitor all your servers in one consolidated view?
- Is product X agent and web based for scalability?
- Does product X use minimum connections to collect monitoring data from your MySQL servers?
- Does product X store monitoring data so you can analyze it later using any BI app?
- Does product X notify you of problems when you are not using the GUI?
- Does product X describe and help you fix/tune what/where/how when needed?
- Does product X plug into your existing notification/alert/escalation system?
- Does product X show you the top-N query resource hogs for specified periods of time?
- Finally, does product X come as part of solution that includes a version of MySQL fully supported with regular updates, and production support provided by the MySQL Support team for both the MySQL server and the monitoring product?
Not sure about other Monitoring products, but the answers for the Enterprise Monitor are, in no certain order, yes yes, yes, yes, yes, yes, yes, yes and yes.
I am also torn when I am asked to compare how the Enterprise Monitor compares to custom in-house written scripts used to monitor MySQL. Knowing the blood, sweat, tears and pride that goes into each script I usually ask "how will you spend your time now that won't have to collect and store metric data and then write, version, or maintain your own scripts to monitor it?" The former DBA in me recognizes when this particular question strikes a chord...
-
When asked about competing MySQL Monitoring products I think it is best to focus most on the value the Enterprise Monitor provides by asking some simple questions:
- Does product X allow you to see and monitor all your servers in one consolidated view?
- Is product X agent and web based for scalability?
- Does product X use minimum connections to collect monitoring data from your MySQL servers?
- Does product X store monitoring data so you can analyze it later using any BI app?
- Does product X notify you of problems when you are not using the GUI?
- Does product X describe and help you fix/tune what/where/how when needed?
- Does product X plug into your existing notification/alert/escalation system?
- Does product X show you the top-N query resource hogs for specified periods of time?
- Finally, does product X come as part of solution that includes a version of MySQL fully supported with regular updates, and production support provided by the MySQL Support team for both the MySQL server and the monitoring product?
Not sure about other Monitoring products, but the answers for the Enterprise Monitor are, in no certain order, yes yes, yes, yes, yes, yes, yes, yes and yes.
I am also torn when I am asked to compare how the Enterprise Monitor compares to custom in-house written scripts used to monitor MySQL. Knowing the blood, sweat, tears and pride that goes into each script I usually ask "how will you spend your time now that won't have to collect and store metric data and then write, version, or maintain your own scripts to monitor it?" The former DBA in me recognizes when this particular question strikes a chord...
-
Friday, July 25, 2008
MySQL Query Analyzer: Finds good code, gone bad
In my 14 years in development I learned that outside of poor schema design, nothing drains the performance of an application more than poorly performing SQL code. Even code that ran well on day one of production would sometimes come back to bite at the worst possible times. Even worse, as a DBA I was consistently asked to bail out a development team that was either tuning their code before the rush to production or that was trying to finger code that had fallen victim to a dropped or changed index. Never fun.
As a Product Manager with MySQL I have learned from meeting with friends/customers that this experience hasn't really changed much since I left the field. I hear things like:
- MySQL is not well instrumented for tracking code level performance metrics
- Logs are OK, but not centralized and too low-level for easy navigation
- We need help identifying "good code gone bad" and "bad code gone worse"
To these friends I say we have listened (and will continue to) and help is on the way! My team has been busy working on the new MySQL Query Analyzer that will ship as part of the Enterprise Monitor a little later this year. Just a few bullets on what it does and what you can do with it.
Cool things it does:
- Proactive collection and identification of problematic SQL code.
- Historical browsing/analysis of queries across all servers.
- Aggregated roll ups of all queries in canonical form (no variables).
- Fully qualified views of worst performing statements.
- Drill downs into query details, number of executions, execution stats
These things are baked into the Monitor to help:
- Tune your SQL code before it is promoted to production
- Quickly identify queries that are sapping performance
-- by query type, content, server or application user
- Once problematic code is surfaced you can:
-- view an EXPLAIN plan to see where you need indexes
Here's a screenshot of the Monitor with Quan enabled:
We are currently working with a few "friendlies" on the first beta, but plan to open things for general beta soon. Stay tuned (no pun intended...), more details to follow on how you can help with this.
As a Product Manager with MySQL I have learned from meeting with friends/customers that this experience hasn't really changed much since I left the field. I hear things like:
- MySQL is not well instrumented for tracking code level performance metrics
- Logs are OK, but not centralized and too low-level for easy navigation
- We need help identifying "good code gone bad" and "bad code gone worse"
To these friends I say we have listened (and will continue to) and help is on the way! My team has been busy working on the new MySQL Query Analyzer that will ship as part of the Enterprise Monitor a little later this year. Just a few bullets on what it does and what you can do with it.
Cool things it does:
- Proactive collection and identification of problematic SQL code.
- Historical browsing/analysis of queries across all servers.
- Aggregated roll ups of all queries in canonical form (no variables).
- Fully qualified views of worst performing statements.
- Drill downs into query details, number of executions, execution stats
These things are baked into the Monitor to help:
- Tune your SQL code before it is promoted to production
- Quickly identify queries that are sapping performance
-- by query type, content, server or application user
- Once problematic code is surfaced you can:
-- view an EXPLAIN plan to see where you need indexes
Here's a screenshot of the Monitor with Quan enabled:
We are currently working with a few "friendlies" on the first beta, but plan to open things for general beta soon. Stay tuned (no pun intended...), more details to follow on how you can help with this.
Thursday, July 24, 2008
To be completely open and honest...
I remember the day I got "the" call. It was in January, 2001 and I was working as a part of an engineering team developing an online, web-based customer billing system for Louisville Gas & Electric. The app was being developed using VB.net on a Windows DNA (distributed network app) architecture that would eventually be served up under IIS. Cutting edge stuff for the day, ancient by today's standards. The call came from my current boss, Robin Schumacher, who wanted me to join him in something call "Product Management" with Embarcadero Technologies. Overall the move has been a good one, but to be completely honest, I miss getting elbows deep in technical details. Along the way, I have had the opportunity to work with some of the best database management tools on the market. I was fortunate to manage Embarcadero's flagship Rapid SQL and DBArtisan products during my 4+ year tenure and had the great privilege of working with one of the highest performing engineering teams in the ISV business.
Robin left Embarcadero to start the Product Management group at MySQL in June, 2005. He recruited me again and I joined him in December of 2005 to manage what was then call MySQL Network. Having worked primarily for corporate IT shops and commercial software companies, it was quite an adjustment coming to work for MySQL. Front and center, the primary product is open source and free to use; hard to swallow for the capitalist in me, but hey I quickly adapted. Actually, I prefer the MySQL model to the old school, licensed, proprietary model. Basically, you use our product to build great apps and when those apps become critical to your business, you pay us to help you make sure they are up and running great. Win-win!
At MySQL I work with some of the best and brightest Engineering and business minds in the industry. My primary partnership is with the Enterprise Tools Engineering team led by Gary Whizin and Mark Matthews. Our primary work is focused on the MySQL Enterprise Monitor which comes as part of a MySQL Enterprise subscription. MySQL Enterprise is our subscription based solution that includes the Enterprise Server and regular bug fixes, the Enterprise Monitor and Expert Advisors and our stellar Production Support Services. You can learn more here. We (PM and the Enterprise Tools team) release new versions of the Monitor 2-3 times a year to ensure that the product continues to evolve. Along these lines we are currently working on Enterprise Monitor 2.0 that will ship with advanced Query collection and analysis capabilities. Of all the product releases I have been involved with since jumping from development to Product Management, this one has me the most excited. It might be because we are addressing a common DBA/Developer pain point with a very real, easy to use solution. It might be because every single customer we have spoken to echoes this pain. It might be those things, but I think I am most excited because some of the proprietary tool vendors in the MySQL monitoring space have creatively adopted our very public facing roadmap and announced similar features in their products. I think this is great! We want them to be successful and to enable more people to adopt and deploy MySQL as part of their move to innovative Web 2.0 technologies. IMHO this is a win all the way around!
I will post more details on the new beta release of the Enterprise Monitor with the new Query Analyzer in my upcoming posts. It is mighty cool!
Robin left Embarcadero to start the Product Management group at MySQL in June, 2005. He recruited me again and I joined him in December of 2005 to manage what was then call MySQL Network. Having worked primarily for corporate IT shops and commercial software companies, it was quite an adjustment coming to work for MySQL. Front and center, the primary product is open source and free to use; hard to swallow for the capitalist in me, but hey I quickly adapted. Actually, I prefer the MySQL model to the old school, licensed, proprietary model. Basically, you use our product to build great apps and when those apps become critical to your business, you pay us to help you make sure they are up and running great. Win-win!
At MySQL I work with some of the best and brightest Engineering and business minds in the industry. My primary partnership is with the Enterprise Tools Engineering team led by Gary Whizin and Mark Matthews. Our primary work is focused on the MySQL Enterprise Monitor which comes as part of a MySQL Enterprise subscription. MySQL Enterprise is our subscription based solution that includes the Enterprise Server and regular bug fixes, the Enterprise Monitor and Expert Advisors and our stellar Production Support Services. You can learn more here. We (PM and the Enterprise Tools team) release new versions of the Monitor 2-3 times a year to ensure that the product continues to evolve. Along these lines we are currently working on Enterprise Monitor 2.0 that will ship with advanced Query collection and analysis capabilities. Of all the product releases I have been involved with since jumping from development to Product Management, this one has me the most excited. It might be because we are addressing a common DBA/Developer pain point with a very real, easy to use solution. It might be because every single customer we have spoken to echoes this pain. It might be those things, but I think I am most excited because some of the proprietary tool vendors in the MySQL monitoring space have creatively adopted our very public facing roadmap and announced similar features in their products. I think this is great! We want them to be successful and to enable more people to adopt and deploy MySQL as part of their move to innovative Web 2.0 technologies. IMHO this is a win all the way around!
I will post more details on the new beta release of the Enterprise Monitor with the new Query Analyzer in my upcoming posts. It is mighty cool!
Subscribe to:
Posts (Atom)
