Another one of the sessions at the MySQL Users Conference I attended was Tuning MySQL5 SQL and Stored Procedures by Guy Harrison from Quest Software. A global company with 6000+ customers.

Guy has written a number of Oracle Performance Books in the past. His work now is on the “Spotlight” product family – Database diagnositic tools converting data to graphical representations. For these products, MySQL 5 and InnoDB only is necessary, simply due to accessing the right internal information for presentation. There are Freeware MySQL product downloads.

In this presentation he stated, nothing he was talking about specifically was relatively new. He did make quite a funny comment, “He is now seeking refugee status in the MySQL Community”.

Guy is author of O’Reilly “MySQL Stored Procedure Programming” Book. I managed to get for free at the conference from the MySQL Quiz night, in addition to a shirt and cap for stumping a Guru.

His talk were on tools and techniques for tuning MySQL.

• Explain Command – reveals what the optimizer intends to do
• Explain Extended

mysql> explain extended select ...;
mysql> show warnings G
Shows what the optimizer actually did. In this example, An IN was converted to EXISTS

There were 4 ways to provide optimizer hints.

1. STRAIGHT_JOIN
2. USE INDEX(…)
3. FORCE INDEX(…)
4. IGNORE INDEX(…)

In addition to the Show Query Log, there are Innodb specific commands, two in particular.

show status like 'innodb%'
* innodb_buffer_pool_read_requests
* innodb_data_read

Indexing and the optimizer

• In MySQL Index is the best tool to improve performance, however sometimes it’s better to access the entire table.
• Indexes generally effective when between 5% and 20% of rows are accessed.
• Subqueries need to be satisified by an index or performance will be quite inefficent.
• Overloading indexes with additional columns when key queries only use a few columns can enable improved performance.

Not all indexes are created equal. In the following examples, each advancement improved performance.

• No indexes ()
• Single Index (customer)
• multiple indexes (customer, product)
• concatenated indexes (customer + product)
• covering index (including required columns, customer+product+qty)

Examples of SQL that can’t benefit from Indexes.

• Derived tables – SELECT table in a from clause, creates a temporary table and will never get an index.
• Views with UNIONS/GROUP BY

A comment from the audience was that derived tables can be of a benefit to a correlated sub-query in specific examples.

Stored Procedures provided a mixed blessing for performance.

• Can improved perfomrance when high network overhead.
• Some improvement on parsing.
• Breaking up complex queries may provide benefits.
• SQL is highly optimized for SET operations.
• SP is not optimized for number crunching. Computionally not a fast language.

A written routine to calculate prime numbers provided the following performance (from most expensive to least) MySQL SP, Oracle SP,PHP,Perl,Java ,VB.NET ,C (gcc). This showed an example that was excessively inefficient. On the other hand, if the program is network dependent (e.g. access a million rows, perform some statisical aggretation). Comparatively the same between Java and SP locally, but much better in a remote host mode.

Performance of SQL in a SP will dominate overall performance. Where SQL is tuned, goto tried and proven traditional optimisation techniques.

• Optimize iterations
• Optimize Logic/Testing
• Avoid recursion

Loop Management

• Only perform necessary code within iterations
• LEAVE or CONTINUE when possible in loops
• Test the most likely IF/THEN statements first
• extract if comparisions duplicated to produce nested if’s (within reason)

Some guidelines for Triggers.

• Triggers will have a non-trival overhead for even the simplest trigger.
• Due to FOR EACH ROW only, don’t have expensive SQL in any trigger.
• Very carefully tune SQL in triggers.
• Empty trigger produced 12% overhead.

For more information check out www.quest.com/mysql

I didn’t take any photos myself at the MySQL User’s Conference. In reflection, perhaps I should. I did make it into some photos however. A few of them here.

Stolen from Jeremy Cole’s Photos at http://jcole.us/gallery/uc2006

Thursday’s Keynote speaker at the MySQL Conference was Mark Shuttleworth talking on The Ubuntu Project: Improving Collaboration in the Free Software World.

Ubuntu from Canonical Ltd is fastly becoming one of the most popular Linux Distributions, based on Debian. Here were some notes from his presentation. Mark had some really powerful one or two line slides that typified both his presentation intention and the goal of Ubuntu.

Delivering on the Promise of Free Software

• MySQL levelled the playing field, individuals could use MySQL to build applications that could now compete in the industry.
• Apache, Python and Linux made it all possible.
• This has provided the Opportunity and passion to build software that can serve a purpose.
• Ubuntu is the continuation of levelling the playing field. Making always available at no cost.
• Whatever your vision, you have the opportunity without financial limitation.
• To be a pioneer in new fields, open source freedom enables the opportunity to anybody.

The New Deal of Free software

* Different Economics

• Ubuntu is world class and free of charge.
• Important to recognise the governence.
• We don’t build 99%, it’s built by communities. We want to maintain good healthy and open relationships with the communities of software developers and contributors.

* Different culture

• Transparency. You can participate, you get to see how the software is built. Difficult for co-ordination of people worldwide, the advantage is you get to be people in the public space everywhere.
• To be able to try, experience to express their own inspiration.

Our vision is to build a complete community-driven distribution that can grow and substain itself through support and services.

Self substaining at a professional level. Community Driven.

• The Ubuntu Proposition
• Genuine Freedom and Flexibility
• Zero Licensing Fees
• Certification: Hardware, Softare, Skills
• Superb Commercial Technical Support
• Standards Compliance
• Internatiolzation and customization
• Collobaration to the Core

We do the best work when we focus on extreme challenges.

Mark made a great comment about the Chinese Version, What was in English on any pages, for Mark was the saving grace, but for the Chinese user, that was the wall, so a committed emphasis on Internationalisation was important.

Ubuntu as you know it today is widely rated #1 desktop Linux with millons of users globally, with complete office and home desktop distributions. We started on the desktop, we are continuing to expand into different variants including the enterprise. The next release in June 2006 – Dappy – 6.06 will includ LTS (Long Term Support) 3 years for Desktop, 5 years for server.

We maintain relationships with the community ecosystem.

• Looking the the User Relationships.
• Prime focus is simplicity. Also Comprehensive and Community, Current.
• For the business relationships, wanting to offering Tiered Partnership Program and Extensive Global Solutions Provider Network.

Free Software Relationships

• Close to upstream (e.g. MySQL, Apache, Firefox etc)
• Close also to other distributions (both debian based and not). We want the best of their work to also be part of what’s available to all.
• Need to ramp up the collaboration of projects especially between projects. Check out launchpad.net
  Site – Much easy to colloborate across projects
• With MySQL and Ubuntu you have freedom to deploy without asking permission, and support when and where you need it
• A key is make packaging ubuntu and mysql the best getting contributions from the community. Expresses most powerfully what MySQL can do.

Beating history into submission

• We judge ourselves by history. We should look at our strengths. we should ask “What futures are enabled.”

Free software enables a different economic paradigm.
Embrace it.

A question from the audience was where does the name Ubuntu come from. The Answer was.
Ubuntu is common to many African languages. It means “human-esse”. The Ubuntu root is common to a lot of words. Some means included “The way we look after other people defines who we are. ”

Kubuntu – Has a meaning “For Humanity”.

My talk at the MySQL Users Conference went well. MySQL For Oracle Developers was part of two talks, the second was by Mark Leith on MySQL For Oracle DBA’s.

I had a number of positive comments from attendees, including Ken Jacobs of Oracle who also contributed some valuable information in comparision of UPDATE/DELETE ORDER BY/LIMIT statements I was unaware of.

You can download my paper here.

My slides, like many talks just scratched the surface in the alloted. I’ve been working on additional reference material, a work in progress is available at MySQL For Oracle Developers. My continuatation on this will be dependent on feedback from the community of it’s intended value.

Everybody can contribute to the MySQL Open Source Community. Here are some ideas:

• You can start with completely a MySQL Conference Survey form (those forms that we are all avoiding).
• You can download, evaluate and test newer versions of MySQL Products.
• You can contribute to the MySQL Forums to ask questions and even give answers.
• Log Bugs & Enhancements on MySQL Products.
• Write about your experiences with MySQL in a Blog and add your Blog to PlanetMySQL. With now over 100 feeds, if everybody wrote something about MySQL in the next year, we would have over 1000 feeds next conference.
• Contribute your thoughts (via wiki), MySQL products, code snippets and examples to MySQL Forge (thanks Mike)
• You can even contribute to the MySQL Source Code.

If you are passionate about MySQL and you are attending the MySQL User Conference, then jump into the Open Source MySQL Community.

At the MySQL User Conference – MySQL Quiz Show we were asked to Stump a MySQL Guru. Well I submitted a few questions, and the first one stumped the guru, being Jeremy Cole, and my prize was a MySQL tee-shirt.

The Question: What are the current supported languages of the GRT Environment used in the MySQL GUI products?
The Answer: Lua, C, C++, Java, Python and PHP

Joe Kottke from FeedBurner in his MySQL Conference presentation today FeedBurner: Scalable Web Applications Using MySQL and Java mentioned a novel way of extracting SQL Select Statements from a MySQL Server.

The obvious MySQL ways include SHOW [FULL] PROCESSLIST, the Slow Query Log, and the General Query Log. In 5.1 you also use the INFORMATION_SCHEMA.PROCESSLIST.

Anyway, Joe mentioned he does a strings on tcpdump to extract the queries, and there was no need to restart the server (this point interested me). So, not letting a challenge go, having never used tcpdump.

Well, here’s how.


tcpdump -w mysqldump.txt -i lo


NOTE: You must log output to file with -w to get the full packets and not just headers.

In a seperate session.


$ mysql -uroot -p mysql -h `hostname`
mysql> select * from user;
mysql> exit
# I did a ctrl-C in the other terminal (not sure if necessary)
$ strings tcpdump.txt
select * from user
mysql
user
user


NOTE: localhost wasn’t valid, it had to be the hostname of the server. Of course you could access the server from a different client.

Of course there may be other or better ways, but it worked for me and was rather cool.

Perhaps this is a good reason to use SSL?

Just to add some more features of the MySQL Migration Toolkit from the MySQL Conference presentation MySQL Migration Toolkit by Mike Zinner from MySQL AB.

You can extend and modify the funcionality of the Migration toolkit with the supplied Eclipse Java project. I’ll need to download the code to write some more detailed notes, but here were a few points from Mike.

Classes are in the com.mysql.grt.modules Package Space
The main configurable classes are nameed Migration????..java and ReverseEngineer???.java with ???? being for the different Database Products

Within the Eclipse project, you can incoporate other languages. For example, ReveserEngineerMySQL Jdbc.java for example has a callGrtFunction to code written in C. This I’m assuming will allow you to incoporate all the other languages of the GRT such as Python, Lua. (I’m still impressed you can actually do this, a mini project for another day to pull this apart.

A question from the audience was about Oracle User Defined data types. MigrationOracle.java for example can easily be extended to cater for the migration of user defined data types to a suitable valid MySQL alternative.

References
MySQL Migration Toolkit – Provides details, screenshots and Flash Tutorials for SQL Server, Oracle and Access Migrations.

It just dawned on me as this topic was mentioned in a MySQL Conference presentation yesterday in a manual process. So my thought is, why can’t the following functionality be added to the MySQL server.

My request for two new extensions to the SHOW STATUS command.

SAVE STATUS – This enables the current SHOW STATUS to be saved (or more specifically cached), you can only keep one copy per server instance.

DIFF STATUS – This shows the difference between the current SHOW STATUS and the last saved show status from the SAVE STATUS command.

This would quickly, easily and interactively via a mysql prompt enable a DBA to see the state of change in a more condensed form. The syntax may need to cater for Session/Global scope, and of course would need to be appropiately named, my examples is just an idea.

Of course you can write an easy script that does the same, and provide a diff, but why not include it directly?

Tuesday’s Keynote speaking from the MySQL Users Conference including a presentation from Greg Gianforte of RightNow Technologies on “Bootstrapping: Starting an Open Source Business With Almost No Money!” Read More.

Taking directly from Greg’s presentation, here are some key points for bootstraping that meant something to me.

• Sales is Job #1
• Don’t spend Beyond Your Means
• There Is Always Another Way
• What is Your Noble Purpose?

Some great points. In summary, not having funds should not be an excuse, find another way . You just need to think creatively. I agree completely, thinking creatively is something I do well. Immerse yourself – talk to lots of people, develop a one page description, get it out to as many people as possible, talk to people, ask for an order. Of course sales is #1, but for me personally, and also within the essense of the presentation – Honesty. His passion about helping people to get started has also lead to publishing a book for bootstrappers (in paperback to save money).

One of the key goals is to “Help our clients to better serve our customers”. Greg was also correct in the need to correct the loss of ethics in the IT and software industry, something I fight for back at home.

A number of these points provided great relevence given I’ve worked for two failed Internet startups.

The closing quiz raised some great questions. Here they are. (Hopefully now reproducible in the Open Source community, oops too late now!)

• #1 What should you do first?
• #2 You have an idea. What next?
• #3 You have a good product, What next?
• #4 You find a prospect. What next?
• #5 Your product does not do everything a prospect wants. Should you?
• #6. A major publication calls saying they are writing a big article about your products and wants you to buy an ad in the same issue of their magzine. You should?
• #7 You are still gettting started and Dun & Bradstreet is asking for company info and detailed financial data. Should you:
• #8 A major prospect says they want to fly to visit you, but you are still working out of your house. Should you:
• #9 You are the only person working full time in the business and you are aksed how many employees you have. Tell them:
• #10 You’ve just made your first sale and generated your first montly profit. You should.

I’ll save the expense by not giving any answers, but my favourite answer was for #9.

Greg started the presentation with the right words for me. Love Mountains, Love Skiing, check out our Careers. Anybody that knows me, knows that my plan for this year, it to be in the Northen Hemisphere for Winter, working for an organisation that is close to the snow, enabling me to go skiing. I just may have check it out.

Last night, I attended Mike Zinner’s presentation of MySQL GUI’s, most specifically MySQL Workbench. In summary I was very impressed with the current work, and the future potential. I’ve been waiting for now about a year to get access to full strength MySQL Workbench (it has been available for some time, but still in early development), having previously used DB Designer for MySQL Data Modelling. One of the limitations is finding a quality tool that runs under Linux. Well MySQL workbench will satisfy this and far more in the future.

A quick summary of some of the glossy features.

• Reverse Engineering of schemas from multiple input sources including MySQL, Oracle, MS SQL Server.
• Synchronisation with a MySQL Database.
• Quick Re-Synchronisation.
• Full GUI tools and features for creating and maintaining database objects including tables, primary keys, foreign keys, views.
• Fully customisable styles via XML and <svg> functionality
• Intitutive point, click and drag functionality for database objects (primary keys for example)
• An interactive shell for the GRT.

The Best Feature

The introduction of the GRT Shell, an interactive shell will provide near unlimited potential.

From the shell you can do everything you can do via the GUI, and you can do it multiple languages including Lua, Python, C/C++ Java and in future Ruby, as well as the potential of many more. The potential of Scripts and Plugins is limitless. Mike demonstated a number of simple examples, so I’ll be very keen now to review and contribute to this functionality now that I’ve seen it in action. MySQL is working actively for community contribution with the recent introduction of the MySQL Forge.

Some of the discussion was the future direction of technology. Presently, there is some limitation of reuse in the end user GUI. A rich GUI is provided, but is platform specific, providing a different look and feel in Linux, Mac OS and Windows. One of the options, and definitely of interest is the consideration of using Java as a standard GUI language.

One of the requests was for more feedback, greater community input and contribution. I agree there needs to be better means here, one of my suggestions was perhaps an interactive Feedback model built into beta products, actively prompting users to provide feedback to MySQL. This would also enable MySQL to gain greater insite into the users of the GUI products.

In summary, excellent work Mike, with a team of just 4 that supports and develops all the MySQL GUI products.

Here is an overview diagram of the MySQL Workbench.

While many people will be blogging about the conference sessions and MySQL features, functionality and sessions, I thought it would be important to raise awareness of the creativity of developers often lost during the product lifecycle.

I met at the Speakers Function last night Jim Starkey, the founder of Netfrastructure, Inc, a company aquired by MySQL recently. At the conference was the official launch of a new transactional storage engine to be included in the MySQL 5.1 release is codenamed “Falcon”.

Too often, the flare and creativity of products, and also the initial history is lost when it reaches the marketing department of an organisation. In this case, after talking with Jim, we decided it was important to ensure the name “Falcon” remains. Why should the name conform to something generic, and quick frankly boring. So please join me.