There has been some discussion recently regarding the death of MySQL Read Replication starting with Brian Aker and then Farhan, Arjen and Paul have all chimed in. Whatever you want to call it, the next generation of replication approach is clearly on the agenda of the industry leaders and pack followers. We should take a programmatic look however and ask ourselves a few questions. Such as:

1. The Use — What is/was MySQL Replication used for?
2. The Reasons — Why was it used?
3. The Problems — Why is there a need for something better, different or improved?

Read Scalability

1. The Use — Clearly the most commonly known method of MySQL Replication is for Read Scalability. The ability for many more reads to occur against the data then writes. I call it the YouTube effect, thousands/millions of reads (i.e. view a video) to a single write (i.e. upload of a video)
2. The Reasons — Database servers could not support the read/write load. Applications were poorly designed for sharding data. The database was being used for a function when it wasn’t really needed. The MySQL Server can’t scale to a massive number of cores.
3. The Problems — MySQL Replication suffers from the asynchronous condition. A successful transaction on the Master, is not confirmed with a Slave before acknowledgment to the client. Google has made inroads in their own environment to improve the asynchronous nature with the SemiSyncReplication patch, now submitted back to the community. The number one effect of asynchronous replication is lag, and again poorly developed applications simply can’t support successful scale-out. Other problems include locking contention when not using a transactional engine.

Backups

1. The Use — A Replication slave is used to perform a database backup, designed to not affect production usage.
2. The Reasons — MySQL does not support a single “unbreakable” online backup solution. Period. There are partial/incomplete implementations, dependent on storage engines and data volume. There is a Roadmap for a solution in MySQL 6.0 (finally), but it’s way too early to tell what features will there be, will you need to pay for some of them, will it just work, let alone who is going to upgrade to 6.0 for this.
3. The Problems — There exists no consistent method for backup across all storage engines. There is no compatibility for backup/point in time recovery (e.g. binary logs/redo logs) across different master/slave servers. No mirrored binary logs is also a problem, something also in the Google Patch.

Failover

1. The Use — You need Business Continuity or Revenue Continuity Solutions (RCS) as discussed by Marc Simony. You need to ensure the maximum uptime possible with a failover infrastructure when some component (software or hardware) fails and your production database is not accessible.
2. The Reasons — MySQL provides no Online DDL. In a 24×7 large volume, changing functionality application this provides an unnecessary complexity for applying changes easily.
3. The Problems — When correctly configured and applied in a controlled situation, failover can work, provided your data is not too large, and your traffic is not too much. It simply doesn’t have a guarantee to work without data loss in an uncontrolled (i.e. disaster recovery) situation.

Others

There are more reasons, something to write about at a later time. These would include.

• Testing
• Alternative data access path (e.g. Data Warehousing)
• Upgrade Path/Migration Path

Options

To handle Read Scalability. Why select something from the database (or various level of caches in the database call when it can be managed by the Application). Memcache is the obvious buzz word here. With tighter database integration (such as the patches and commands FaceBook has) this could be a winner.

Backups is a tough one, without practical storage engine agnostic consistent solutions I don’t feel the backup conundrum will be resolved anytime soon.

To handle a successful failover you need a guarantee that the data is consistent. A Synchronous Replication solution will solve this. MySQL Cluster is a synchronous solution, however it is impractical for basically every application that is already running with MySQL Replication.

What other options exist? Food for thought.

About Memcache

A little on Memcache for those still in the dark ages. Many large sites such as FaceBook,Fotolog, Wikipedia and Slashdot are strong proponents of Memcached. Created originally for Live Journal Memcached has become the defacto standard in this level of application data caching.

The description from the source.

memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

This week I’d expected to hear good friend Giuseppe (CCO) Maxia speak about MySQL SandBox at the Hamburg MySQL April Meetup.

This is product I’ve thought about using, wanted to use, but just never got to the point to using. I download the current version 1.18, I had MySQL tar’s already of 5.0, 5.1 and 6.0 and was all ready until the late topic change.

However due to the language barrier in the second talk, I got a one-on-one lesson. Now I know how it works, and understands the strengths I can use it as part of my standard vanilla testing. There are some improvements I could see, something perhaps I can contribute if they allow a Perl part-timer too. The joy of open source is I can add, modify, change and submit my work, if not wanted for example, there is simply no loss to me.

• By default installation goes to $HOME, and then creates a different standard for individual, replication or multiple instances. I’d like to see this default to a directory say $HOME/msb so all different installs are all under one directory node. You can specific this currently via –sandbox_directory otherwise.
• express_install.pl could benefit from a –help option
• I’d like the set_replication.pl and set_many.pl commands to support an array of mysql tar’s. I’m most interested in testing replication say from 5.0 to 5.1 and 6.0, not just 5.0 to 5.0. I’d also like to be able to install a 5.0, 5.1 and 6.0 versions and be able to use the multi_cmd to run the same SQL across different versions easily.
• The multi_cmd would benefit from a means to list the identify of the instance with the output. It’s hard as MySQL does not have MYSQL_SID, only a server_id which can be a little confusing.

Thanks Giuseppe for your work on this product.

Current syntax

./install.pl --help
Unknown option: help
    The MySQL Sandbox,  version 1.18 05-Apr-2008
    (C) 2006,2007,2008 Giuseppe Maxia, MySQL AB
syntax: ./install.pl [options]
    -h --home_directory = name          The home directory. (default: $HOME (/Users/rbradfor))
    -d --sandbox_directory = name       Where to install the sandbox, under home-directory
    -P --sandbox_port = number          The port number to use for the sandbox server.
                                        (Default: 3310)

    --datadir_from = name               Where to get datadir files from. Available options are
                                        archive   will be taken from the archived data
                                                  provided with the package. They include
                                                  default username and passwords
                                                  ( DEPRECATED )
                                        script    the script mysql_install_db is called, with
                                                  default users, no passwords.
                                        dir:name  will be copied from an existing mysql directory
                                        (Default: script)

    -i --install_version = name         Which version to install (3.23, 4.0, 4.1, 5.0 or 5.1) default: 5.0
    -b --basedir = name                 Base directory for MySQL (default: /usr/local/mysql)
    -m --my_file = name                 which sample my-{small|large|huge}.cnf file should be used
                                        for additional configuration
                                        You may enter either the label (small|large|huge) or a full
                                        file name. (default: none)

    -f --conf_file = name               Configuration file containing options like the ones
                                        you can give on the command line (without dashes)

    -U --operating_system_user = name   Operating system user (for mysql installation)
                                        default: $USER (rbradfor)

    -u --db_user = name                 user for global access to mysql (Default: sandbox)
    -p --db_password = name             password for global access to mysql (Default: sandbox)
    -c --my_clause = name               option to be inserted in a my.cnf file
                                        it may be used several times

    --prompt_prefix = name              prefix to use in CLI prompt (default: mysql)
    --force                             Use this option if you want to overwrite existing directories
                                        and files during the installation. (Default: disabled)

    --no_ver_after_name                 Do not add version number after sandbox directory name (default: disabled)
    -v --verbose                        Use this option to see installation progress (default: disabled)
    --load_grants                       Loads the predefined grants from a SQL file.
                                        Useful when installing from script.
                                        (default: disabled)

    --no_load_grants                    Does not loads the predefined grants from a SQL file.
                                        (default: disabled)

    -t --interactive                    Use this option to be guided through the installation process (default: disabled)
    --no_confirm                        suppress the confirmation request from user

Example:
     ./install.pl --my_file=large --sandbox_directory=my_sandbox

Tonight’s Hamburg MySQL Meetup was a packed room of 60-70 people at Sun in Hamburg. The most I’ve seen @ a MySQL Meetup across three cities, countries, continents and years (my first Oct 5, 2004).

As expected most was in German. Pity, I would have liked to hear about Open Office and MySQL. Giuseppe, CCO (That’s Chief Cartoon Officer – I liked that one) gave in English a general review of MySQL. He highlighted the reasons “Why did Sun by MySQL?” including most popular, most dynamic and people with the Freedom to work anywhere.

Three points I would like to re-iterate for the community.

The MySQL Forge contains the Worklogs, i.e. what is going to be developed in current & future versions and the status of tasks in next version, say in review, in development, completed development etc. This is important because this gives exposure to the world what MySQL are indeed working on.
• The MySQL University each Thursday gives a free session that includes from an expert audio and visual slides as well as the ability to ask questions via IRC. I’ve attended one of these before and I’d like to see this continue to improve and grow.
• MySQL is open and values external contributions from the community. My new co-worker Paul creator of PBXT highlights this. I hope with additional time within the MySQL Community I can help to contribute and lobby for greater differentiation of MySQL Community and MySQL Enterprise in favor of the Community.

Giuseppe did however not talk about the MySQL SandBox as scheduled. I was even prepared having wanted to use his software previously. I’d downloaded and was all ready. I did manage to get a one-on-one lesson which I found most useful, and a few points of enhancements to consider.

I didn’t know there were so many MySQL people in the area, the following attending tonight. Lenz, Giuseppe, Kai V, Kay, Jan, Ulf W and Georg whom I’d not met before. Most I’ll see next week at the MySQL Users Conference.

I had need to show the official MySQL business shirt to somebody, so I did a Google Search on “mysql conference photos” (without the quotes) as I know I’ve seen photos of staff at the conference online before. Google Search Link

To my utter surprise, Google provided 3 image search results. Two of them were of Paul McCullagh, CTO of PrimeBase Technologies. Not only was this unexpected, the first image of Paul was actually from my own Blog. Go figure how I managed that Search Engine Optimization.