Support the MySQL Community Photo Day

Updated On good advice from Sheeri I made a few comments clearer.

It has been proposed that the integral “MySQL Community Photo Day” be on Thursday April 17 2008, the final day of the MySQL Users Conference.

Wear a t-shirt from an open source community project on Thursday, whether a PrimeBase PBXT one or your favorite open source project. Get your photo taken with the masses of community supporters. If your not attending this year, this doesn’t mean you can’t also contribute a photo yourself from what location you are from. Save a saving fund for next year but get us a photo.

It doesn’t have to just be Thursday, photos will be accepted at any time before then. Upload a photo and win a prize. (Baron you definitely get chocolate)
Their is a Flickr group called “MySQL Community Photo Day” at http://www.flickr.com/groups/mysqlcommunityphotoday

This is a chance for you to support the community. There is already a small band of supporters before my post already including Colin C, Paul M, Lenz G, Baron S, Sheeri K.

Replication++, Replication 2.0, Replication.Next

There has been some discussion recently regarding the death of MySQL Read Replication starting with Brian Aker and then Farhan, Arjen and Paul have all chimed in. Whatever you want to call it, the next generation of replication approach is clearly on the agenda of the industry leaders and pack followers. We should take a programmatic look however and ask ourselves a few questions. Such as:

  1. The Use — What is/was MySQL Replication used for?
  2. The Reasons — Why was it used?
  3. The Problems — Why is there a need for something better, different or improved?

Read Scalability

  1. The Use — Clearly the most commonly known method of MySQL Replication is for Read Scalability. The ability for many more reads to occur against the data then writes. I call it the YouTube effect, thousands/millions of reads (i.e. view a video) to a single write (i.e. upload of a video)
  2. The Reasons — Database servers could not support the read/write load. Applications were poorly designed for sharding data. The database was being used for a function when it wasn’t really needed. The MySQL Server can’t scale to a massive number of cores.
  3. The Problems — MySQL Replication suffers from the asynchronous condition. A successful transaction on the Master, is not confirmed with a Slave before acknowledgment to the client. Google has made inroads in their own environment to improve the asynchronous nature with the SemiSyncReplication patch, now submitted back to the community. The number one effect of asynchronous replication is lag, and again poorly developed applications simply can’t support successful scale-out. Other problems include locking contention when not using a transactional engine.

Backups

  1. The Use — A Replication slave is used to perform a database backup, designed to not affect production usage.
  2. The Reasons — MySQL does not support a single “unbreakable” online backup solution. Period. There are partial/incomplete implementations, dependent on storage engines and data volume. There is a Roadmap for a solution in MySQL 6.0 (finally), but it’s way too early to tell what features will there be, will you need to pay for some of them, will it just work, let alone who is going to upgrade to 6.0 for this.
  3. The Problems — There exists no consistent method for backup across all storage engines. There is no compatibility for backup/point in time recovery (e.g. binary logs/redo logs) across different master/slave servers. No mirrored binary logs is also a problem, something also in the Google Patch.

Failover

  1. The Use — You need Business Continuity or Revenue Continuity Solutions (RCS) as discussed by Marc Simony. You need to ensure the maximum uptime possible with a failover infrastructure when some component (software or hardware) fails and your production database is not accessible.
  2. The Reasons — MySQL provides no Online DDL. In a 24×7 large volume, changing functionality application this provides an unnecessary complexity for applying changes easily.
  3. The Problems — When correctly configured and applied in a controlled situation, failover can work, provided your data is not too large, and your traffic is not too much. It simply doesn’t have a guarantee to work without data loss in an uncontrolled (i.e. disaster recovery) situation.

Others

There are more reasons, something to write about at a later time. These would include.

  • Testing
  • Alternative data access path (e.g. Data Warehousing)
  • Upgrade Path/Migration Path

Options

To handle Read Scalability. Why select something from the database (or various level of caches in the database call when it can be managed by the Application). Memcache is the obvious buzz word here. With tighter database integration (such as the patches and commands FaceBook has) this could be a winner.

Backups is a tough one, without practical storage engine agnostic consistent solutions I don’t feel the backup conundrum will be resolved anytime soon.

To handle a successful failover you need a guarantee that the data is consistent. A Synchronous Replication solution will solve this. MySQL Cluster is a synchronous solution, however it is impractical for basically every application that is already running with MySQL Replication.

What other options exist? Food for thought.

About Memcache

A little on Memcache for those still in the dark ages. Many large sites such as FaceBook,Fotolog, Wikipedia and Slashdot are strong proponents of Memcached. Created originally for Live Journal Memcached has become the defacto standard in this level of application data caching.

The description from the source.

memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

The MySQL SandBox

This week I’d expected to hear good friend Giuseppe (CCO) Maxia speak about MySQL SandBox at the Hamburg MySQL April Meetup.

This is product I’ve thought about using, wanted to use, but just never got to the point to using. I download the current version 1.18, I had MySQL tar’s already of 5.0, 5.1 and 6.0 and was all ready until the late topic change.

However due to the language barrier in the second talk, I got a one-on-one lesson. Now I know how it works, and understands the strengths I can use it as part of my standard vanilla testing. There are some improvements I could see, something perhaps I can contribute if they allow a Perl part-timer too. The joy of open source is I can add, modify, change and submit my work, if not wanted for example, there is simply no loss to me.

  • By default installation goes to $HOME, and then creates a different standard for individual, replication or multiple instances. I’d like to see this default to a directory say $HOME/msb so all different installs are all under one directory node. You can specific this currently via –sandbox_directory otherwise.
  • express_install.pl could benefit from a –help option
  • I’d like the set_replication.pl and set_many.pl commands to support an array of mysql tar’s. I’m most interested in testing replication say from 5.0 to 5.1 and 6.0, not just 5.0 to 5.0. I’d also like to be able to install a 5.0, 5.1 and 6.0 versions and be able to use the multi_cmd to run the same SQL across different versions easily.
  • The multi_cmd would benefit from a means to list the identify of the instance with the output. It’s hard as MySQL does not have MYSQL_SID, only a server_id which can be a little confusing.

Thanks Giuseppe for your work on this product.

Current syntax

./install.pl --help
Unknown option: help
    The MySQL Sandbox,  version 1.18 05-Apr-2008
    (C) 2006,2007,2008 Giuseppe Maxia, MySQL AB
syntax: ./install.pl [options]
    -h --home_directory = name          The home directory. (default: $HOME (/Users/rbradfor))
    -d --sandbox_directory = name       Where to install the sandbox, under home-directory
    -P --sandbox_port = number          The port number to use for the sandbox server.
                                        (Default: 3310)

    --datadir_from = name               Where to get datadir files from. Available options are
                                        archive   will be taken from the archived data
                                                  provided with the package. They include
                                                  default username and passwords
                                                  ( DEPRECATED )
                                        script    the script mysql_install_db is called, with
                                                  default users, no passwords.
                                        dir:name  will be copied from an existing mysql directory
                                        (Default: script)

    -i --install_version = name         Which version to install (3.23, 4.0, 4.1, 5.0 or 5.1) default: 5.0
    -b --basedir = name                 Base directory for MySQL (default: /usr/local/mysql)
    -m --my_file = name                 which sample my-{small|large|huge}.cnf file should be used
                                        for additional configuration
                                        You may enter either the label (small|large|huge) or a full
                                        file name. (default: none)

    -f --conf_file = name               Configuration file containing options like the ones
                                        you can give on the command line (without dashes)

    -U --operating_system_user = name   Operating system user (for mysql installation)
                                        default: $USER (rbradfor)

    -u --db_user = name                 user for global access to mysql (Default: sandbox)
    -p --db_password = name             password for global access to mysql (Default: sandbox)
    -c --my_clause = name               option to be inserted in a my.cnf file
                                        it may be used several times

    --prompt_prefix = name              prefix to use in CLI prompt (default: mysql)
    --force                             Use this option if you want to overwrite existing directories
                                        and files during the installation. (Default: disabled)

    --no_ver_after_name                 Do not add version number after sandbox directory name (default: disabled)
    -v --verbose                        Use this option to see installation progress (default: disabled)
    --load_grants                       Loads the predefined grants from a SQL file.
                                        Useful when installing from script.
                                        (default: disabled)

    --no_load_grants                    Does not loads the predefined grants from a SQL file.
                                        (default: disabled)

    -t --interactive                    Use this option to be guided through the installation process (default: disabled)
    --no_confirm                        suppress the confirmation request from user

Example:
     ./install.pl --my_file=large --sandbox_directory=my_sandbox

MySQL Speakers and Presenters at LinkedIn

There is a Linked In group I created some time ago but forgot to advertise that is for MySQL Speakers and Presenters.

If you a speaker or presenter of MySQL content, confirm your registration here.
You will need to have a reference to a website confirming you have been a speaker at a MySQL Event such as a User Conference, MySQL Camp or Local MySQL Users Group.

Hopefully overtime we can build a consolidated index at MySQL Forge Presently some pages exist including MySQL Conference & Expo and User Group Presentations but I’m accepting input for a model to have a central page, and link to or upload of presentation. Any input welcome.

Storage Engines at the MySQL Conference

I’ll be following closely the progression of Storage Engines available in the MySQL Database server, well soon to be available when 5.1 gets to GA (hopefully by end of Q2 which is what we have been told). Tick, Tick, time is running out.

PrimeBase XT (PBXT) and Blob Streaming is a focus as PrimeBase Technologies, the company which I want to note for people is an Open Source company, committed at providing an open source alternative to the other commercial players. You also have at the MySQL Conference talks on the the existing InnoDB from Innobase (a subsidiary of market RDBMS leader Oracle). There is a Nitro presentation, an Infobright presentation, no Solid presentation surprisingly (the IBM news happening after submissions closed). We also have from MySQL, presentations on the internally developed storage engines Falcon and Maria, both products that won’t even be in 5.1 but 6.0, however Maria is presently a different branch of 5.1 so I don’t know how that works. Will it be in 5.1?

But what I want to seek is more news of KickFire, a Diamond Sponsor, an engine with embedded H/W, something that’s been obviously worked on in reasonable stealth. For me it’s not just interesting, it’s a competitor in our technology space, so I’ve been researching Joseph Chamdani and some of his patents.

Plenty of news in the past few weeks on Kickfire including Kickfire Update by Keith Murphy on April 3, Kickfire: stream-processing SQL queries by Baron Schwartz on April 4, Kickfire looking to push MySQL limits by Farhan Mashraqi on April 4, and Kickfire Kickfire Kickfire by Peter Zaitsev on April 4, and myself back on March 23.

So what can I make from the lack of company information and posted information to date.

  • Hardware based acceleration.
  • No Solid State Drive (SSD) Technology, at least not yet but C2App mentions SSD.
  • Data Warehousing, lending to thinking it’s not a transactional storage engine
  • A new storage engine and a new approach to data storage. I find this surprising, as it takes years to develop a feature complete storage engine, and most new 5.1 storage engines are indeed existing products, take Nitro, Solid, Infobright and Falcon. Only PBXT has been written from the ground up for MySQL 5.1, so looking to know more about it’s development
  • Expensive, it’s dedicated H/W + (assuming) MySQL Enterprise + Storage Engine

When is a website useless?

Well this is one way. Time Magazine has so much advertising garbage, the page on my Macbook shows only 16 words of meaningful article text on the page. Those being almost rather useless as well:

“Wired News Senior Editor Kevin Poulsen and a small team of editors do a great job”

Hamburg MySQL Meetup – Apr 2008

Tonight’s Hamburg MySQL Meetup was a packed room of 60-70 people at Sun in Hamburg. The most I’ve seen @ a MySQL Meetup across three cities, countries, continents and years (my first Oct 5, 2004).

As expected most was in German. Pity, I would have liked to hear about Open Office and MySQL. Giuseppe, CCO (That’s Chief Cartoon Officer – I liked that one) gave in English a general review of MySQL. He highlighted the reasons “Why did Sun by MySQL?” including most popular, most dynamic and people with the Freedom to work anywhere.

Three points I would like to re-iterate for the community.

    The MySQL Forge contains the Worklogs, i.e. what is going to be developed in current & future versions and the status of tasks in next version, say in review, in development, completed development etc. This is important because this gives exposure to the world what MySQL are indeed working on.

  • The MySQL University each Thursday gives a free session that includes from an expert audio and visual slides as well as the ability to ask questions via IRC. I’ve attended one of these before and I’d like to see this continue to improve and grow.
  • MySQL is open and values external contributions from the community. My new co-worker Paul creator of PBXT highlights this. I hope with additional time within the MySQL Community I can help to contribute and lobby for greater differentiation of MySQL Community and MySQL Enterprise in favor of the Community.

Giuseppe did however not talk about the MySQL SandBox as scheduled. I was even prepared having wanted to use his software previously. I’d downloaded and was all ready. I did manage to get a one-on-one lesson which I found most useful, and a few points of enhancements to consider.

I didn’t know there were so many MySQL people in the area, the following attending tonight. Lenz, Giuseppe, Kai V, Kay, Jan, Ulf W and Georg whom I’d not met before. Most I’ll see next week at the MySQL Users Conference.

A heartfelt apology

While I mentioned in Not Stuck, “thanks to those concerned”, perhaps I did not illustrate clearly that I was suitability moved by the genuine concern of close friends regarding my plight, and I did really appreciate the comments and initial feedback.

I’ve talked with close friends such as Jay, Frank, Marc and tonight both Giuseppe and Sheeri and while I’ve been told it’s a good one, and we laughed and smiled I do value the genuine friends I have in the MySQL Community.

My apologies for any personal pain cause during April 1st. I hoped that I’d replied via email to each commenter promptly to minimize any additional delay.

And sorry Marten, for the lost productivity as I’ve heard about, I guess I owe you one.

Using Vodafone UMTS in Germany

These instructions apply to the Novatel MC950D UMTS USB modem running under Mac OS/X 10.5 (Note these instructions do not apply to OS/X 10.4)

1. Software Installation

You first need to install the software as supplied on CD.

This will give you a Vodafone Mobile Connect folder including the Vodafone Mobile Connect application and manuals in various languages.

2. Connect your modem

Insert your data SIM card (which should be a different SIM from your mobile phone) into the modem and connect to our Mac. You should get a consistent blinking RED light.

3. Run the Connect Application

Selecting the Vodafone Mobile Connection Application you will get a program that will search for a network connection. You will prompted to enter your SIM card pin.

When an appropriate connection has been made, your modem should now have blinking GREEN light in intervals of 3.

You need to select the [Activate] Button at the bottom right of the window.
Your modem will now have a blinking GREEN light in intervals of 2.

Select the Support tab will provide some technical details of the modem and OS.

4. Getting a Network Connection

Open System Preferences then select Network.

Click the [+] button to add a new network type.

Select from the dropdown list the Interface Notavel and give the service an appropriate name. NOTE: It does not appear you can change the name at a later time.
Click the [Create] button.

Now back on the Network page, click the [Advanced] Button to change settings.

Select the WWAN tab. Select the Vendor Notavel from the supplied list.

NOTE: When selecting a different Vendor the Model list does not change. You need to Select [Ok] then [Apply] and then re-enter the WWAN panel.

Selecting the Model GSM you will be prompted with two additional fields. Enter WEB.VODAFONE.DE in the APN field (This is very important).

Select the DNS tab. No changes should be necessary. This should remain empty.

Select the WINS tab. No changes should be necessary. This should contain the NetBios Name entered, other fields empty.

Select the Proxies tab. No changes should be necessary.

Select the PPP tab. No changes should be necessary.

5. Making the connection

You are now ready to go.

Click the shiny [Connect] button. If all goes well you should end up with a connection.

6. Checking Settings

Following a successful connection, the [DNS] tab shows updated information.

Unexpected SEO results

I had need to show the official MySQL business shirt to somebody, so I did a Google Search on “mysql conference photos” (without the quotes) as I know I’ve seen photos of staff at the conference online before. Google Search Link

To my utter surprise, Google provided 3 image search results. Two of them were of Paul McCullagh, CTO of PrimeBase Technologies. Not only was this unexpected, the first image of Paul was actually from my own Blog. Go figure how I managed that Search Engine Optimization.

Come and get a t-shirt at UC2008

Here I am at my desk sporting the PrimeBase supporters t-shirt that will be available at the exhibitors booth at the 2008 MySQL Conference. The front is rather uneventful with the official logo, but the back will be worth the experience. So everybody interested in supporting PBXT as the transactional storage engine for MySQL developed by the community and for the community, please come and see us and mention the secret password.

We have been placed way back in the right hand side of the exhibitors hall at booth 518, in front of the Open Source and OEM providers.