You are here: Home / Archives for 2008

Log Buffer #94: a Carnival of the Vanities for DBAs

by

April 25th, 2008 – by Ronald Bradford

Welcome to the 94th edition of Log Buffer, the weekly review of the database blogsphere. Adding to the list of usual database suspects, I have some more alternative considerations for our readers this week.

We start with Conferences

Still some discussion from last weeks’ 2008 MySQL Conference & Expo.

Baron “xarpb” Schwartz calls it correct in Like it or not, it is the MySQL Conference and Expo. Matt Assay of c|net gives us some of his opinions in three posts Two great posts on MySQL, Back to the future for MySQL and Between two consenting corporations… in followup to last week’s active slashdot discussion. Many others have also commented if you have not been following the news released before opening keynotes.

If you didn’t get a hard copy, Sheeri Kritzer Cabral has published the Pythian EXPLAIN Cheatsheet many attendees received.

Also last week was Collaborate 08 – Technology and Applications IOUG forum for the Oracle Community.

This week we also see the Web 2.0 San Francisco in action, and excitement is also brewing for the PGCon – PostgreSQL Conference for Users and Developers happening in under a month as Robert Treat has Plane tickets booked for PGCon. Postgres was also visible at the MySQL Conference & Expo if you were looking with a prominent consulting team downing the blue elephant during the event. Wish I’d taken a photo now!

Still more news from Adam Machanic of the Pythian group with SQLTeach Toronto: Almost Here.

Common threads

The 2008 Google Summer of Code announced this week showcases the Open Source databases MySQL (14 projects) and PostgreSQL (6 projects). Kaj Arnö talks more in Fourteen Summer of Code projects accepted 2008. The company PrimeBase Technologies also features strongly with two projects for the Blob Streaming storage engine for MySQL as I detail in Media Blob Streaming getting a Google boost.

MySQL

DTrace Integration with MySQL 5.0 – Chime demo in MySQL Users Conference 2008 by Jenny Chen is an example of Sun’s Open Source contribution to MySQL which I saw as a physical demo last week. Unfortunately, due to the imbalance in actually getting new functionality into Community contributions (actually non existence in current or next mysql version :-(), this functionality is only really for show. Dtrace with MySQL 6.0.5 – on a Mac describes some of this work actually making it into the next, next version. It seems this next Falcon Preview is available but not announced by MySQL generally as I note in Continued confusion in MySQL/Sun release policy.

MySQL Gurus Mark Callaghan and Brian Aker comment respectively here and here on MySQL Heap (Memory) Engine – Dynamic Row Format Support. Work submitted by Igor Chernyshev of eBay Kernel Team (whom I’ve met previously and was most impressed with his ability to submit MySQL patch work, with little previous MySQL kernel knowledge, but extensive C++ knowledge). This work also contributed to eBay Wins Application of the Year at MySQL Conference & Expo.

Mark also mentions in his post “How do users get it? There is no community branch into which people can submit changes with a GPL license.“. A topic your’s truly has also mentioned regarding the Community contributions, development and release. Perhaps a sign of more benefit to the community soon as Monty mentions.

Baron Schwartz comments on Keith “a.k.a Kevin” Murphy’s work in Spring 2008 issue of MySQL Magazine. With a quick plug also for his upcoming book “High Performance MySQL – Version 2″ (me giving it a plug also now), Baron also has the best published anti-spam sniffer email I’ve seen, and recently updated to his new employer. Check his blog and let me know.

Postgres

Joshus Drake of Command Prompt Inc. The Postgres Company gets excited in Is that performance I smell? Ext2 vs Ext3 on 50 spindles, testing for PostgreSQL and gives us some insight into different settings of two popular file system types. It would be great to see a follow up with a few more different filesystems types.

Pabloj “so many trails … so little time” extends his MySQL example to Postgres in Loading data from files. And on Postgres Online Journal, we get An Almost Idiot’s Guide to PostgreSQL YUM giving you a step by step guide of PostgreSQL setup, including the all important “Backing up Old Version”.

Oracle

We get a detailed book chapter from Keith Lake of Oracle OLAP The most powerful, open Analytic Engine in his extensive post on Tuning Guidance for OLAP 10g. David Litchfield brings attention in A New Class of Vulnerability in Oracle: Lateral SQL Injection. The title is sufficient for all Oracle DBA’s to review.

Don Seiler gives his experience in Bind Variables and Parallel Queries Do Not Mix when an Oracle Bug is discovered the database to 64-bit H/W..
Matching LOB Indexes and Segments by Michael McLaughlin gives us a good CASE/REGEX SQL example exam question, and simple output to monitor the growth of LOBs in your Oracle database.
Additional readings for Oracle folks can be found with Kenneth Downs writing Advanced Table Design: Resolutions and Dan Norris’ Collaborate 08 thoughts gives a concise review of a largely attended Oracle event.

SQL Server

B Esakkiappan’s SQL Thoughts gives us a throughout lesson on SQL Server 2005 Database Transaction logs with Know the Transaction LOG – Part – 1, Part – 2, Part – 3 and Part -4 Restoring Data.

Paul S. Randal of SQL Skills adds Conference Questions Pot-Pourri: How to create Agent alerts to his writings following many requests after a recent workshop.

In Scalability features I would like to have in SQL Server Michael Zilberstein lists 3 key features including “Active-Active cluster”, “Indexes per partition” and “Bitmap indexes and function based indexes”.

Ingres, Times Ten, Google App Engine and more

Some movement in the Ingres world with Deb Woods of Ingres Technology Blog discussing in Inside the Community – Ingres style…. the Ingres Engineering Summit occurring this week. Attendees included newbies to a 24 year Ingres veteran. That beats my experience in Ingres which now extends 19 years.

We get another very detailed installation description, this time for Times Ten in Install Oracle TimesTen In-Memory Database 7.0.4 on Linux.

Just a few weeks ago, a new database offering hit the market with the Google App Engine. News this week includes
Google App Engine Hack-a-thons! being announced with events in New York on May 7th and San Francisco on May 16th. As a developer with an account and an excuse to use it more, I can’t win, being in the right towns on the wrong dates.

OakLeaf Systems this week writes Comparing Google App Engine, Amazon SimpleDB and Microsoft SQL Server Data Services. Another good read just for comparison.

Not in a blog, but in discussion in at the recent MySQL, was msql. It was interesting to find out that PHP was originally developed for msql first, and only used MySQL as the preferred database after some functionality requirement. Interesting what could have been?

In Conclusion

Thanks Dave for the opportunity to contribute to the week in review. Until my chance to charm the readers next time.

I leave you with a photo, and challenge our readers to find another person who would be capable of wearing a t-shirt that states “My free software runs your company”. Michael Widenius- Founder and original developer of MySQL can, and my thanks to you for MySQL, and the Vodka shots at the Conference last week.

Happy Earth Day 2008!

Making business decisions for the community and the enterprise

by

I was prompted following a few key words by Marten Mickos at the Sun Dinner on Wednesday evening, and subsequent one on one discussion with Marten, to post my thoughts of some significant news this week announced at the MySQL Conference. The decision to provide as it’s been termed is “Enterprise only features”. It is unfortunate this was not discussed in Marten’s opening keynote, having been exposed the evening before in the Partner’s meeting and hitting the blog sphere before the conference officially started.

MySQL, past, present and future as an Open Source company requires a functional business model to succeed. This includes the funding of resources and the technology progression. It is also necessary in this business climate to build a successful business quickly. How do you do this? Well that’s probably the difference between a successful CEO and an unsuccessful one, and what Marten Mickos has produced is clearly very successful.

I may not necessarily agree with the decisions made, more specifically not understanding at this time the rationale of which features are free and which are likely to be commercial, but I respect the decision made by Marten Mickos. These new feature considerations are in a future release of MySQL, they are also I’m sure not yet set in stone, however MySQL can not be all things to all people, no software can. It reminds me of the Homer Simpson car, designed to do everything Homer wanted in a car, but it bankrupted the previously successful company due the views of one individual to solve all their own needs, but not the needs of the majority. Who is affected by this decision, who will benefit, again it’s too early to tell.

Monty indicates this is a MySQL decision, not a Sun decision. This indicates the transition of MySQL to being under the Sun banner of the largest open source company is well, still in transition. Today it was again confirmed to me, that the MySQL database will always be GPL and MySQL will not never revoke the functionality that powers the world’s largest websites as free software for the database server.

As an advocate for the MySQL community, I’d like to consider myself one of the pulses, a thought in the MySQL conscious and even a vocal lobbyist. I am however not interested in being a disruptor in the MySQL ecosystem. A Communications Lesson on Slashdot I believe correctly states “Sun to Begin Close Sourcing MySQL.” The headline is wrong.

A number of people have posted their comments, let’s stop bickering about it, and let’s see something positive happen for the benefit of the community. For example, as I’ve mentioned previously regarding the lack of differentiation for the Community version, again mentioned by Mark Callaghan in A better (community) HEAP engine where a worthwhile patch can’t be of benefit to the community in a binary release for the lay person.

When in management, I am responsible for contributing to the success of the company, to play a significant role in the functional business model, to ensure funding for resources and technology progression. In other terms, how can revenue generation be achieved to fund prominently salaries of staff, including my own. How am I going to do this? This will be the difference between my huge success or not.

Continued confusion in MySQL/Sun release policy

by

In review of some list posts today, I came across the Falcon Preview 6.0.5 downloads available from the MySQL Forge (even that is unclear, but the directory indicates this on the forge). The Forge Wiki Documentation indicates the 6.0.5 release features (without a download link), however the official MySQL downloads for 6.0, and the directory structure for all MySQL releases only describes 6.0.4.

Nothing in the MySQL News and Events of this new release. The documentation for 6.0.5 is also unclear as it lists this version as Not Yet Released.

This comes on the heels of “Pending General Availability/near-final release candidate” (See here and here), and the Press Releases for Sun Celebrates Third-Party MySQL Storage Engines only to be negated by Sun Microsystems Announces MySQL 5.1 which broke the plugins as they were not compatible.

I don’t wish to create confusion, but it’s unclear what is happening here, not withstanding the new preview only has 5 builds and excludes Mac OS/X which means I can’t natively test.

Media Blob Streaming getting a Google boost

by

The 2008 Google Summer of Code MySQL Projects are now available. MySQL has 14 listed projects, one of the ~190 different Open Source products listed. Unfortunately there is no summary to see the total number of projects being sponsored across all products.

Media Blob Streaming actually has the luxury of two approved projects, so they have plenty of mentoring work at PrimeBase Technologies.

Raj Kissu Rajandran will be working on BLOB Streaming Support for phpMyAdmin and KishoreKumar Bairi on Streaming Enabled MySQL Driver for PHP. Welcome to world of open source for your respective projects.

The database frontier

by

Jay’s opening lines regarding the final MySQL Conference keynote speaker was: “I work with a lot of data. I think peta-bytes, maybe exa-bytes”. This was relating to Jacek Becla from the Stanford Linear Accelerator Center, giving his presentation on “The Science and Fiction of Petascale Analytics”.

The goal of the Large Synoptics Survey Telescope (LSST) is the storage of 50+ PB of images and 20+ PB data.
Let’s just clarify the size. 20 PB of data = 20 years of HD Movies = 2000 years of 128kb MP3

The next database frontier is obviously building huge databases. What part will MySQL or other relational databases play? Some interesting facts were.

  • The Digital Universe Created 161 Exabytes of data last year.
  • Google, processes 20 petabytes of data per day.

The Operational plan for LSST Project Timeline is 10 years, only starting in 2014. The timeline:

  • 2009 Choosing Technology
  • 2010-2014 constructions
  • 2014-2023 production

The primary goals are: Scale, parallelize, fault tolerant.

Q: What a MySQL fellow does?

by

A: Maria, an ACID, MVCC engine that plans to be the default non-transactional and default transactional engine for MySQL.

Presently development with a team of 6 people and plans of adding 2-3 developers the work on Maria should see the 1.5 release this month.

It was great to here Monty say “We have a policy of zero MySQL Bugs, like the old MySQL way.”

Maria Version History
1.0 – “Crash Safe” — part of a existing 5.1 branch
1.5 – “Concurrent insert/select” to be merged as part of formal MySQL 6.0 release
2.0 – Transactional and ACID
3.0 – High Concurrency & Online Backup
4.0 – Data Warehousing

The schedule has all of the features to be available for the next MySQL Conference Q2 2009

Some points of note:

  • This is a MyISAM replacement.
  • It was interesting to hear about log file size (suggesting being big like 1G), and there are not circular. New log files will be created, and only files purged only when no longer used.
  • There has been a change in default page size, presently defaulting to 8K for both data and indexes.
  • Maria 1.5 does not support INSERT DELAYED and FullText and GIS indexes are not crash safe.
  • There are extensive tests in the MySQL Test Suite
  • Will support READ COMMITTED and REPEATABLE READ (available in 1.5)
  • Every part of the development and process is open an available documentation (unlike some other storage engines Monty mentions)
  • Have a drop everything policy on new bugs to have Maria as stable as possible.
  • Check out the blog at monty-says.blogspot.com

Tips from the MySQL Conference

by

What would be great if people could create a single line (one tip) from each talk and we could aggregate these for an executive summary for tech people.

This was prompted from only a few minutes looking in on Baron Shwartz’s EXPLAIN presentation. What I didn’t know was.

EXPLAIN EXTENDED SELECT …; SHOW WARNINGS; gives the rewritten SQL query

If only I had time to whip out an application on my Google AppEngine and get twitter feeds with say a mysqlconf keyword. Perhaps we need a all night BoF hackfest to do it.

PrimeBase PBXT/Blob Streaming BoF – What you missed.

by

A small but committed group met at 8:30pm to hear more about our the plans from PrimeBase Technologies here at the 2008 MySQL Conference. Our discussion started in true MySQL form.

Monty Widenius presents to the group plastic cups and a bottle of Absolut Vodka.
After a shot, Paul starts with “While I can still talk”.
Monty, slams another bottle of Vodka on the table.
We all laugh.

Paul outlined some of the roadmap plans from existing the Alpha release to Beta releases.
He talked about the plans for Synchronous Replication and there was active discussion on various use cases.
There was also discussion and input on Solid State Drive (SDD) Technology which will be tested with PBXT in the coming months.

Scaling Wisdom

by

The 20 second summary from the Scaling MySQL – Up or Out? from our panel of experts at 2008 MySQL Conference and Expo.

  • Paul Tuckfield from YouTube — The answers to everything is replication, you just have to rephrase the question.
  • Jeff Rothschild from Facebook — Memory, the source of all problems is your developers.
  • Domas for Wikipedia — You should be afraid that 10 min structural change may answer detailed problems.
  • Fahan Mashraqi from Fotolog — Architect property, the most optimized schema may not be enough, what is the cost of serving the data, no just the time to run the SQL.
  • John Allspaw from Flickr — There is nothing more permanent then a temporary solution
  • Monty Taylor from MySQL — You have to know what’s happening on every piece of your technology stack.

What's in a new name

by

Also in the MySQL Press Releases today but dated for tomorrow is Sun Microsystems Announces MySQL 5.1.

I find the wording clearly a new language from my previous understanding — “pending general availability of MySQL™ 5.1″.

We now see the trademark notice, obviously a Sun influence.
We now have a “pending” GA version. MySQL is obviously very keen to release MySQL 5.1 for GA. This was expected at last years’ MySQL conference. Many in the past year have expected this prior to this year’s conference (we are now being informed late Q2 2008). There was an anticipation there would be two RC versions, this is now the third. So what exactly does “pending” mean? Will 5.1.24 be renamed Production if it passes community acceptance (I say community because it’s not an Enterprise release). This would be a change from previous naming policy. What’s most likely is hopefully they release 5.1.25 as production. Comes back then to why the words “pending general availability”, and not “next release candidate” which is what it is.

Previously MySQL also made new with the initial RC status of 5.1, moving away from the previously policy.

The next opening keynote – Everything fails, All the Time

by

Our third keynote this morning was by Dr Werner Vogels – CTO Amazon.com

His second question to the audience “How many of you don’t shop at Amazon?” When one or two people raised their hands he commented, “Can I talk to you later.”

He was here to talk about Amazon as the technology company, forget about shopping. Think about the technology that drives that. There were some large and impressive numbers about Amazon. The one stood out was 3.9M shipments for peak day (didn’t mention which day)

Some points from the session.

  • Some Big News — Persistence Storage for Amazon EC2
  • Amazon started as technology consumer, we are now a technology provider.
  • We were enterprise scale, now we are at web scale.
  • Taking the approach, get big fast, the result was every architectural principle was broken.
  • Moving into Software as a Service
  • Develop -> Test -> Operate
  • There is a box between Test and Operate (undifferentiated heavy lifting)
  • We seem to put data centers in the same places as trailer parks.
  • At Amazon we expect our data centers to fail.
  • We expect our software to tolerate these fails.
  • We want to be able to loss a data center and not have this affect our customers.
  • Amazon, brings data centers down multiple times a year, just to show the software survives.
  • The provider model — Scalable, Cost Effective, Reliable, Simple
  • Addressing Uncertainty.

He used the 365 main power failure example. Large Web 2.0 Web sites were down including: Live Journal, Craigslist, Technorati, TypePad, Yelp and USA Today.

Is Amazon a viable option in these times? Worthy of consideration.

Updated You can view more links including video courtesy of Sheeri Kritzer at MySQL Forge Conference Notes

Standing room only

by

At Day 1 of the 2008 MySQL Conference and Expo today, our high numbers of attendees (reported at 2,000) have resulted in Standing Room only in a lot of talks. This has got to be excellent PR.

I got to sit in on the Memcached and MySQL session by Brian Aker and at the end I stuck my head in to Best Practices for Database Administrators by and Explain demystified by Baron Schwartz both 2008 MySQL Award winners.

All full to overfilling presentations.

PrimeBase XT (PBXT) in the news

by

In today’s Official MySQL Press Release, PrimeBase XT (PBXT) has been named along with three other storage engine partners in “Sun Celebrates Third-Party MySQL Storage Engines“.

This a great achievement for a small company to be recognized in the certified storage engine partner program, in comparison to the other companies that are much larger on the balance sheet. This continues the news with last week you may have read PrimeBase Technologies a MySQL Platinum Level Partner.

A noted absence from the list is Nitro, an expected absence was Solid.

Off to a flying start

by

Marten has opened the 2008 MySQL Conference & Expo. This time he started in his opening comments “I have more to say to more people, and given less time to say it”.

His answer to why Sun bought MySQL included slides showing “Alignment in Culture and Vision” and “What’s in it for you – Performance & Scale, Support, Marketplace”.

This year the MySQL Conference has over 2,000 people and 55 exhibitors.

What was funny, was the photo showing the burning of the IPO Prospectus. Marten mentioned now with many Sun lawyers he has to be more careful what to day. I actually have an interesting extension to this at Watching what you say

Some points of note for me:

  • The Web Economy continues to have exponential growth and the need for new technology but a goal of linear growth.
  • Continuing to mention becoming “Disruptive Online Innovations”
  • The Online landscape consists of the Software Development Model, Business Model,Software Deployment Model and Organisational Model.
  • He Reiterated the Design Priorities at MySQL. Reliability, performance, easy of Use
  • MySQL Workbench is now GA. Congratulations Michael Zinner.
  • On Storage Engine Update, PrimeBase, PBXT and Paul McCullagh got a mention. In addition to he usual suspects, but ScaleDB & Tokutek are newly mentioned and are Exhibitors this year.

A few words from Jonathan Schwartz

by

Following Marten Mickos, the second opening keynote at the 2008 MySQL Conference and Expo was by Jonathan Schwartz CEO and President of Sun Microsystems. Blog

His opening joke was about dinner with Marten, to which Marten said “You not going to get a keynote, unless you buy the company.”

So what was striking for me in his presentation “What is Sun’s Agenda?”

  • There is no open-source phone yet, but that’s an industry that needs disrupting.
  • Like the need for water or electricity, The Network Has Become A Social Utility.
  • We want to work with the community, create greater innovation.
  • The future, the price tag of Free, the philosophy of Freedom

I had a chance to meet Jonathan and Rich Green on Sunday night, and it was great to see Jonathan learning about, and getting behind the product PBXT – The Community Engine a MySQL 5.1 open source engine.

MySQL Awards at the MySQL Conference & Expo

by

Announced this morning is this year’s MySQL award winners.

2008 MySQL Application of the Year

  • FaceBook – Social Network
  • Virgin Mobile France – Mobile Operator
  • eBay – ECommerce Site

2008 MySQL Partner of the Year

  • Zmanda
  • Microsoft
  • Computer center

2008 MySQL Community Member of the Year

Watching what you say?

by

Marten has opened the 2008 MySQL Conference and Expo this morning in Santa Clara.

What was funny in the early slides was the photo showing the burning of the IPO Prospectus. Marten mentioned now with many Sun lawyers in the audience he has to be more careful what to say.

This morning while coming down for breakfast, a Sun employee entered the lift. I introduced myself, and he indicated he knew me by name. When I asked what department he was in, he said “legal”. Being intrigued as to who he knew me, I’d discovered he has read my emails and blog posts.

I must admit I’ve met a number of new people this week and the first word has been “Oh!”, as in they have heard of my name previously.

These have been unexpected responses and information for me, I’m not normally surprised like this. I’ll not be changing what I say, and how I say it, professionally my writing and publishing will continue to embody “Opinions, Expertise, Passion”.

PrimeBase Technologies a MySQL Platinum Level Partner

by

As per the Platinum Partners List PrimeBase Technologies has made a key achievement. I think this is a great list to make for the company. There are only 18 Platinum partners, and PrimeBase Technologies are the third Storage Engine provider and the second first open source storage engine company.

The News Release quotes “As a key milestone in our relationship with both the MySQL Community and MySQL, this partnership strengthens our commitment and involvement in working with the MySQL database server starting with PBXT and following with additional product developments.”

Update Thanks to Ken Jacobs, I need make a correct, we are not the first “open source storage engine company”, Innodbase, a wholly owned subsidiary of Oracle was indeed.

What is your max_connections?

by

Kevin Murphy just posted Tip of the Day — max_connections and I thought it might be nice to know what people actually set this to so I’ve created a Survey “What’s your max_connections” (totally anonymous).

For the record, while a consultant the record for a production system was 9,500. Yes you read right, and I might add the client was running a 32-bit OS as well, just to boot.
Apparently they had also tested they could get 9,500 connections, problem is they didn’t do anything with the connections, so didn’t see any per thread memory usage, and I did not get confirmation of how the actually system memory usage was used during the test.

Share/Add This Buttons on sites

by

I’ve noticed a change of buttons lately on sites where you can bookmark/share the relevant information. So I’ve done a cross sample of A Computer Site – www.dell.com, The MySQL Acquirer – www.sun.com, A News site – www.cnn.com and a technology information site – www.techcrunch.com

What got me to go back and research is I’d never seen a FaceBook icon before, or perhaps I’d never paid enough attention.

At the end most sites now wash out to a site called Add This.










MySQL Community Photo Day Prizes

by

I forgot to mention in Support the MySQL Community Photo Day that PrimeBase Technologies is providing 3 prizes for the best photos uploaded.

First Prize $150 Amazon Gift Voucher
Second Prize $100 Amazon Gift Voucher
Third Prize $50 Amazon Gift Voucher

So, take your photo with other community supporters — they’ll also be wearing their open source t-shirts. You can upload photos to www.flickr.com/groups/mysqlcommunityphotoday.

Hey Jake, what's your problem

by

So, I’m checking out the upcoming weather for the MySQL Conference for next week, and Yahoo Weather throws me a 404 Not Found error from their own site. Interesting, what I did particularly like was the URL, it had jake/404.html. What’s the problem Jake?

Where's the News?

by

It’s been six weeks since the completion of the acquisition of MySQL by Sun Microsystems. In this time, on the MySQL News and Press Releases there has only been one (1) Press Release here.
In the six weeks between the announcement and completion of the acquisition MySQL made ten (10) Press Releases, and the six weeks preceding this including the Christmas break there was (six) Press Releases.

On the Sun Press Releases in this period there has been twenty three (23), but just the same one MySQL specific.

The MySQL Developer Zone has picked up three articles in the past week, but nothing prior to that for a month.

So, Where’s the News of what’s happening with MySQL?

It seems our only options are the official company blog of Kaj Arno and his travels , Robin Schumacher and Giuseppe Maxia as other sources of information.

The pursuit of a synchronous world

by

Well at least your MySQL database world.

As Paul eluded to, PrimeBase Technologies has a project to provide synchronous replication for MySQL in a High Availability environment. It is more then an idea, there is a plan.

Is it possible?
What are the use cases?
How can you use it?
Would you use it?

Some input to date. We need these questions and more, and we seeking more input for discussion.

Unfortunately the opportunity to hear any input during a presentation is left to the last day of the conference, so there has been created a BoF session on Tuesday night for a round table discussion if necessary. People are encouraged to bring specific cases and situations for feedback, the reasons why MySQL Replication, MySQL Cluster, DRBD/HeatBeat or any other solution does not satisfy your needs, and what would.

If you can’t wait you may need to seek out Booth #518, and make a time to seek out the Technology Expert.

Just today, Peter Zaitsev of MySQL Performance Blog also writes in State of MySQL Market and will Replication live ?, “Customers are constantly asking me if there is something which would help them to scale MySQL and get some HA out of the box even on the medium level. Seriously – MySQL Cluster, Continuent, Master-Master Replication, DRBD or SAN based HA architectures all have their limits which makes neither of them used for very wide class of applications.” to confirm this pursuit.

Support the MySQL Community Photo Day

by

Updated On good advice from Sheeri I made a few comments clearer.

It has been proposed that the integral “MySQL Community Photo Day” be on Thursday April 17 2008, the final day of the MySQL Users Conference.

Wear a t-shirt from an open source community project on Thursday, whether a PrimeBase PBXT one or your favorite open source project. Get your photo taken with the masses of community supporters. If your not attending this year, this doesn’t mean you can’t also contribute a photo yourself from what location you are from. Save a saving fund for next year but get us a photo.

It doesn’t have to just be Thursday, photos will be accepted at any time before then. Upload a photo and win a prize. (Baron you definitely get chocolate)
Their is a Flickr group called “MySQL Community Photo Day” at http://www.flickr.com/groups/mysqlcommunityphotoday

This is a chance for you to support the community. There is already a small band of supporters before my post already including Colin C, Paul M, Lenz G, Baron S, Sheeri K.

Replication++, Replication 2.0, Replication.Next

by

There has been some discussion recently regarding the death of MySQL Read Replication starting with Brian Aker and then Farhan, Arjen and Paul have all chimed in. Whatever you want to call it, the next generation of replication approach is clearly on the agenda of the industry leaders and pack followers. We should take a programmatic look however and ask ourselves a few questions. Such as:

  1. The Use — What is/was MySQL Replication used for?
  2. The Reasons — Why was it used?
  3. The Problems — Why is there a need for something better, different or improved?

Read Scalability

  1. The Use — Clearly the most commonly known method of MySQL Replication is for Read Scalability. The ability for many more reads to occur against the data then writes. I call it the YouTube effect, thousands/millions of reads (i.e. view a video) to a single write (i.e. upload of a video)
  2. The Reasons — Database servers could not support the read/write load. Applications were poorly designed for sharding data. The database was being used for a function when it wasn’t really needed. The MySQL Server can’t scale to a massive number of cores.
  3. The Problems — MySQL Replication suffers from the asynchronous condition. A successful transaction on the Master, is not confirmed with a Slave before acknowledgment to the client. Google has made inroads in their own environment to improve the asynchronous nature with the SemiSyncReplication patch, now submitted back to the community. The number one effect of asynchronous replication is lag, and again poorly developed applications simply can’t support successful scale-out. Other problems include locking contention when not using a transactional engine.

Backups

  1. The Use — A Replication slave is used to perform a database backup, designed to not affect production usage.
  2. The Reasons — MySQL does not support a single “unbreakable” online backup solution. Period. There are partial/incomplete implementations, dependent on storage engines and data volume. There is a Roadmap for a solution in MySQL 6.0 (finally), but it’s way too early to tell what features will there be, will you need to pay for some of them, will it just work, let alone who is going to upgrade to 6.0 for this.
  3. The Problems — There exists no consistent method for backup across all storage engines. There is no compatibility for backup/point in time recovery (e.g. binary logs/redo logs) across different master/slave servers. No mirrored binary logs is also a problem, something also in the Google Patch.

Failover

  1. The Use — You need Business Continuity or Revenue Continuity Solutions (RCS) as discussed by Marc Simony. You need to ensure the maximum uptime possible with a failover infrastructure when some component (software or hardware) fails and your production database is not accessible.
  2. The Reasons — MySQL provides no Online DDL. In a 24×7 large volume, changing functionality application this provides an unnecessary complexity for applying changes easily.
  3. The Problems — When correctly configured and applied in a controlled situation, failover can work, provided your data is not too large, and your traffic is not too much. It simply doesn’t have a guarantee to work without data loss in an uncontrolled (i.e. disaster recovery) situation.

Others

There are more reasons, something to write about at a later time. These would include.

  • Testing
  • Alternative data access path (e.g. Data Warehousing)
  • Upgrade Path/Migration Path

Options

To handle Read Scalability. Why select something from the database (or various level of caches in the database call when it can be managed by the Application). Memcache is the obvious buzz word here. With tighter database integration (such as the patches and commands FaceBook has) this could be a winner.

Backups is a tough one, without practical storage engine agnostic consistent solutions I don’t feel the backup conundrum will be resolved anytime soon.

To handle a successful failover you need a guarantee that the data is consistent. A Synchronous Replication solution will solve this. MySQL Cluster is a synchronous solution, however it is impractical for basically every application that is already running with MySQL Replication.

What other options exist? Food for thought.

About Memcache

A little on Memcache for those still in the dark ages. Many large sites such as FaceBook,Fotolog, Wikipedia and Slashdot are strong proponents of Memcached. Created originally for Live Journal Memcached has become the defacto standard in this level of application data caching.

The description from the source.

memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.