You are here: Home / Archives for Other

MySQL Predictions for 2007

by

I’m interested to know what people consider will behold MySQL in 2007?

The announcement of “You” as Time person of the year can only considered a huge boost to the opportunities in 2007. So, in 2007 here are my 7 (in no significant order).

  1. 2007 will be the year of the storage engine. We will see 5 offerings for transactional storage engines, 20+ available and practical engines for management of some form of data.
  2. 2007 will see MySQL 5.1 GA (finally).
  3. 2007 will see MySQL release it’s own Falcon Storage Engine (GA not until Q4 :-().
  4. The MySQL Winter of Code will enable the contributions of the community to change feature development. I foresee a Bounty system from an external party or parties for MySQL Features emerging.
  5. MySQL will make major press inroads to the RDBMS Big 3 of Oracle, SQL Server and IBM DB2.
  6. Despite efforts of MySQL AB, major installations of MySQL 4.0 and 4.1 including large ISP’s will hamper the uptake of 5.0 and 5.1 and the de-commissioning of 4.x
  7. A major country government will make an announcement to move to Open Source across servers and desktops, and MySQL will contribute to being an enterprise database offering in systems replacements as part of a longer term strategy.

NY Tech Meetup

by

Tonight I headed to the NY Tech Meetup organized by the CEO of Meetup and co-founder of Fotolog, the company my friend Frank works for.

This forum provided for quick presentations by new NY high tech ventures and other interesting discussions, then enabling further networking between people.

A Perfect Thing


The first speaker was Steven Levy, mentioned on the site as Newsweek’s tech editor & all-around geek writer extraordinaire. He is the author of “The Perfect Thing”, a story of the Apple iPod. He shared a funny story of a dinner where he was seated with Bill Gates at a Microsoft XP launch in late 2001, in which he had just that week got his initial iPod following the launch. When he gave it to Bill Gates, he observed as he described this mind meld as a votex between Bill’s brain and the iPod while he checked it out, exploring all the menu options, buttons and options. 45 seconds later came the comment of something like, looks great, and it works with a Macintosh.

Urbis

Our second speaker was Steve Spurgat from www.urbis.com. The blub. Urbis is a creative community with three types of users: creative people, those who love and support creative people, and those who have opportunities for creative people. It’s very creative.. Some of the interesting features of this site included:

  • Can pre-define the people that can review your submissions, by various criteria, meaning that your feedback can be restricted.
  • You can specify your specific goals for your submission.
  • You can select the present opportunities for your submission.
  • There is an economy system to see reviews of your own work you must review others

Presently only writing is available, but plans for Music, Art and Film will be available in the next few months. With some 12,000+ members and 13% active, it’s a good start.

There was also discussion of copyright, Urbis being a registered copyright agent complying with government guidelines, and of revenue models including the option for fees from publishers, and the potential of ad copy. A competitor site Trigger Street was also mentioned, started by Kevin Spacey.

One Web 2.0 thing I liked about this site, and the next was that the website was the presentation (no powerpoint), and while talking the home page of the website was displayed and the content was dynamically changing, in this case, reviews being submitted online. A good selling point.

LinkStorms

Scott Kolber of LinkStorms was our next presenter. Described as the next generation of links for the web providing context specific fast links and specific navigation from a button, images, banner ad.

The revenue model is CPM plus a publishers setup, maintenance and support fee structure. Apparently up to 40% click thru rate, which is extraordinary compared to the current stats of < 1% for general banners.

When asked what was different with this model, the answer was "the results. It's a better user experience looking at ads".

You can see it in action at Premiere Magazine – The Departed.

CogMap

Brent Halliburton and his approach to a wikipedia of Organization charts with CogMap certainly got the best response the crowd. A good comedian, Brent made the mistake with a slow Internet connection to demonstrate interactively with an example from the audience and not his own prepared content. It ended up not rendering, then crashing but he managed to turn it around into a plus and the best applause of the night.

His idea provoked a wide range of comment and feedback and when asked why? “Because if your an entrepreneur you do things”. “In the big scheme of things I don’t have all the answers. I just put it out there.”

uPlayMe

David Fishman provided the last presentation of uPlayMe, a Windoze program that provides a slant on the community social networking via enternaintment, specifically when they are actually playing via Windows Media Player for example. It’s designed to help people discover other people with the same interests, or weird interests. Some other sites mentioned in the discussion included Last.fm, Pandora and MOG.

2007 Predictions

We ended with an audience participated 2007 predications. The included:

  • No Predication – (The first person from the Board of Advisors I believe that was specifically asked)
  • IP TV market and integration with the TV
  • Will see a Billion $ organization from the NY community
  • The buzz of radios that can do multiple gigibits of transfer between neighours (yes it sounded weird)
  • Era of the connected home, Computer, TV, Stereo
  • Some political thing at change.net
  • Another political thing, an organic style camp debrief
  • The Term 2.0 will cease being used in 2007
  • Skype will be a source of major innovation
  • NY will produce a billion dollar Internet company

Pluggable Storage Engines – What is the potential?

by

I started this post a month ago, but after Kaj’s discussion on the same topic at the MySQL Camp I figured it was time to post.

I had dinner with a friend recently (a very smart friend), and our conversation lead him to ask “What’s different with MySQL?”. One of the things I tried to describe was the “Pluggable Storage Engine Architecture” (PSE) potential for the future that I expect will set MySQL apart from all other Open Source and even commercial databases.

Here are some details of the example I tried to provide, given somebody who understands enough of the general principles of RDBMS’s.

Consider the ability that information (intelligent data) is available within a Relational Database via the appropriate tools and language (e.g. SQL) but it is not physically constrained to Tables, Columns and Rows of data and an application to manage that data which is the present traditional approach. Let’s use images that you take with your digital camera as an example.

In a typical RDBMS application you would create an application to managed the content of your data, with a number of tables, and links to the images etc. Of course you would need an application as well to both view and manage this information.

What if, you simply pointed your database to a directory of images and then was able to query information such as photos by date, or by size, by album, from a certain location, with given keyword etc. Most of this information about digital photographs is already there. This information is encoded into an Exif format that is embedded within JPEG images.

So what’s missing from this information? Tags and Comments are the most obvious, because this information can’t be determined electronically, this is something that humans do. If you could also embedded this information into an image with a suitable tool then you could be ready to manage your photos.

A further extension would be to have Image Analysis capabilities that enabled you to search for photos that contained the sky, or people, or something that was the color red.

What if in the future, your camera’s had a built in GPS and this information recorded within Exif, and then the ability to extend your output to link to popular on line mapping software such as Google Maps would exist. You could then use your digital camera to track your moves, taking photos that could then plot your path over a holiday, and also enabling location based queries.

It was interesting to postulate what ideas may be possible in the futre. I suspect that it won’t be long before we actually see this. So what are the other potentials that you may not consider. Another example may be a MP3 Dukebox style PSE, managing all the information held with ID tags of MP3 allowing you to do with music what could be done with images.

References

Exif Example

Here is some example content of Exif using EXIF Tool

 ./exiftool ~/Desktop/2006_02_23_AirShow/IMG_5966.JPG
ExifTool Version Number         : 6.50
File Name                       : IMG_5966.JPG
Directory                       : /home/rbradfor/Desktop/2006_02_23_AirShow
File Size                       : 2 MB
File Modification Date/Time     : 2006:09:24 17:44:32
File Type                       : JPEG
MIME Type                       : image/jpeg
Make                            : Canon
Camera Model Name               : Canon EOS 300D DIGITAL
Orientation                     : Horizontal (normal)
X Resolution                    : 180
Y Resolution                    : 180
Resolution Unit                 : inches
Modify Date                     : 2006:02:23 16:01:56
Y Cb Cr Positioning             : Centered
Exposure Time                   : 1/320
F Number                        : 10.0
ISO                             : 200
Exif Version                    : 0221
Date/Time Original              : 2006:02:23 16:01:56
Create Date                     : 2006:02:23 16:01:56
Components Configuration        : YCbCr
Compressed Bits Per Pixel       : 3
Shutter Speed Value             : 1/320
Aperture Value                  : 10.0
Max Aperture Value              : 3.5
Flash                           : No Flash
Focal Length                    : 18.0mm
Macro Mode                      : Unknown (0)
Self-timer                      : 0
Quality                         : Fine
Canon Flash Mode                : Off
Continuous Drive                : Single
Focus Mode                      : AI Focus AF
Canon Image Size                : Large
Easy Mode                       : Manual
Digital Zoom                    : Unknown (-1)
Contrast                        : +1
Saturation                      : +1
Sharpness                       : +1
Camera ISO                      : n/a
Metering Mode                   : Evaluative
Focus Range                     : Not Known
AF Point                        : Manual AF point selection
Canon Exposure Mode             : Program AE
Lens Type                       : Unknown (-1)
Long Focal                      : 55
Short Focal                     : 18
Focal Units                     : 1
Max Aperture                    : 3.6
Min Aperture                    : 22
Flash Activity                  : 0
Flash Bits                      : (none)
Zoom Source Width               : 3072
Zoom Target Width               : 3072
Color Tone                      : Normal
Focal Plane X Size              : 23.22mm
Focal Plane Y Size              : 15.49mm
Auto ISO                        : 100
Base ISO                        : 200
Measured EV                     : 9.00
Target Aperture                 : 10
Target Exposure Time            : 1/318
Exposure Compensation           : 0
White Balance                   : Auto
Slow Shutter                    : None
Shot Number In Continuous Burst : 0
Flash Guide Number              : 0
Flash Exposure Compensation     : 0
Auto Exposure Bracketing        : Off
AEB Bracket Value               : 0
Focus Distance Upper            : -0.01
Focus Distance Lower            : 5.46
Bulb Duration                   : 0
Camera Type                     : EOS Mid-range
Auto Rotate                     : None
ND Filter                       : Unknown (-1)
Self-timer 2                    : 0
Bracket Mode                    : Off
Bracket Value                   : 0
Bracket Shot Number             : 0
Canon Image Type                : IMG:EOS 300D DIGITAL JPEG
Canon Firmware Version          : Firmware Version 1.1.1
Camera Body No.                 : 0930402471
Serial Number Format            : Format 1
File Number                     : 159-5966
Owner's Name                    :
Canon Model ID                  : EOS Digital Rebel / 300D / Kiss Digital
Canon File Length               : 2387078
WB RGGB Levels Auto             : 1726 832 831 948
WB RGGB Levels Daylight         : 0 0 0 0
WB RGGB Levels Shade            : 0 0 0 0
WB RGGB Levels Cloudy           : 0 0 0 0
WB RGGB Levels Tungsten         : 0 0 0 0
WB RGGB Levels Fluorescent      : 0 0 0 0
WB RGGB Levels Flash            : 0 0 0 0
WB RGGB Levels Custom           : 0 0 0 0
WB RGGB Levels Kelvin           : 0 0 0 0
Color Temperature               : 5200
Num AF Points                   : 7
Canon Image Width               : 3072
Canon Image Height              : 2048
Canon Image Width As Shot       : 3072
Canon Image Height As Shot      : 2048
AF Points Used                  : Mid-left
Preview Quality                 : Normal
Preview Image Length            : 278318
Preview Image Width             : 1536
Preview Image Height            : 1024
Preview Image Start             : 2108760
Preview Focal Plane X Resolution: 3443.9
Preview Focal Plane Y Resolution: 3442.0
User Comment                    :
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 3072
Exif Image Length               : 2048
Interoperability Index          : R98 - DCF basic file (sRGB)
Interoperability Version        : 0100
Related Image Width             : 3072
Related Image Length            : 2048
Focal Plane X Resolution        : 3443.946
Focal Plane Y Resolution        : 3442.017
Focal Plane Resolution Unit     : inches
Sensing Method                  : One-chip color area
File Source                     : Digital Camera
Custom Rendered                 : Normal
Exposure Mode                   : Auto
Scene Capture Type              : Standard
Compression                     : JPEG (old-style)
Thumbnail Offset                : 2560
Thumbnail Length                : 7680
Image Width                     : 3072
Image Height                    : 2048
Aperture                        : 10.0
Drive Mode                      : Single-frame shooting
Flash                           : Off
Image Size                      : 3072x2048
Lens                            : 18.0 - 55.0mm
Preview Image                   : (Binary data 278318 bytes, use -b option to extract)
Preview Image Size              : 1536x1024
Scale Factor To 35mm Equivalent : 1.6
Shooting Mode                   : Program AE
Shutter Speed                   : 1/320
Thumbnail Image                 : (Binary data 7680 bytes, use -b option to extract)
WB RGGB Levels                  : 1726 832 831 948
Blue Balance                    : 1.140108
Circle Of Confusion             : 0.019 mm
Focal Length                    : 18.0mm (35mm equivalent: 27.9mm)
Hyperfocal Distance             : 1.67 m
LV                              : 14.0
Lens                            : 18.0 - 55.0mm (35mm equivalent: 27.9 - 85.3mm)
Red Balance                     : 2.075767

Zune or zzz?

by

An interesting article in the local New Jersey paper this week. “Will Microsoft iPod-rival Zune be a ‘zoom’ or more of a ‘zzz’. This is Microsoft’s attempt to finally case in the 1.5 billion songs that have been sold online bt the Apple iTunes store. A comment from the article.

The Zune’s wireless function also taxes battery life. And Zun users – Zunies? – are in for some surprises when the actually try zapping songs to each other.
Those songs will deactivate in three days,or after three plays – whichever comes first.

I had a friend also tell me that the new Zune is not compatible with the upcoming Windows Vista. Now if that’s actually true, that’s amazing.

The desire for Performance SQL Tips

by

It seems, people are clammering for a more consolidated help guide for SQL Performance tips.

Jay Pipes at the MySQL Camp ran a session Interactive Top 10 SQL performance Tips. There was plenty of input and discussion, and at the time Sheeri simply typed them into a wiki page for later work.

Well it seems even that rough list is popular at Del.icio.us ranking near the top of the Hot List on the front page. I saw it earlier and it was second or third, but didn’t think of taking a screen shot until now, but it’s still high.

I’d say that we could easily get the Top 10 for up to 10 different categories rather easily. Good luck Jay.

The Falcon!

by

Some early notes by Brian Aker on Falcon as discussed at the MySQL Camp.

Falcon is a transactional engine MySQL will be introducing. The first discussions were held about 3 years ago with Ann Harrison and about 1 1/2 years ago, MySQL started taking seriously the possibilities.

Falcon is not an InnoDB replacement. It’s a different way of looking at the problem of how it looks at and manages transactions, and how it’s designed. It flips around the way data is stored. Some points:

  • It uses as much memory as possible, like Oracle SGA or InnoDB pool.
  • It has a row cache not a page cache for more optimal memory use.
  • No locking at all. Jim doesn’t believe in it for concurrency control. It has total versioning.
  • Falcon has to keep all changes in memory, so not great for user transactions that may take longer
  • Characteristics – Well optimised for short fast web transactions, Designed for environments with lots of memory.

In general discussions is was mentioned from the floor the fear that there will be so many storage engine options, and you will need a matrix for what is good for what.

In conclusion, Brian mentioned it will be alpha before the end of year.

MyISAM++

by

Monty gave us a quick overview of next generation of MyISAM. It is set to include:

  • New data disk format
  • Transaction support
  • multi-versioning
  • row level locking and escalation to table level locks. (interesting)
  • bitmap indexes and new table scanning optimizing indexes with up to 1000x times performance.

No details of time frame were given for delivery, however development is well underway.

Doxygen Project

by

What the?

Well this is the inheritance diagram of the Item Class in the MySQL 5.1 Source tree, nicely documented using the Doxygen tool as mentioned by Jay in his presentation at MySQL Camp.

Jay started the Community Doxygen Project on the Forge to improve the level of commenting enabling a better platform for the community to contribute MySQL server code changes.

At this early stage David Shrewsbury is working on fine tuning initial documentation examples for QA and review. You can check out the Status Page of automated commenting conversion.

You can see the present documentation of MySQL 5.1 source here.

Return to Google Lobby – Camp Photo

by


Early on Sunday Day #3, I dragged a few willing participants out for a “different photo” based on the umbrellas in each Lobby. It worked out well. Special thanks to Kynan who ran around to other lobby’s to find additional umbrellas. (He is the one holding the white one, and yes, that’s a utility Kilt).

I’ll be uploading more in this series to My Flickr Photos MySQL Camp 01 soon.



You can get a larger copy of image Here.

MySQL Replibeertion

by

MySQL Replibeertion was the last scheduled session on Day 2, but not withstanding there was free beer (a lot of), there was a serious side with a Replication Discussion.

One of the first questions by Jeremy was “Are the any big replication users?” to which Sheeri quickly replied “Are you calling me fat again”.

This was a highly interactive session, here are some of the points from the audience.

Some Uses of Replication

  • Backup
  • Hot standby
  • Scaling
  • Data Warehousing
    • Slaves are larger then your database
  • For no special reason
  • Consolidation of multiple sources
  • Support for multiple indexes

Issues

  • Can break
  • replication lag
  • bi-directional replication is not supported
  • hard to setup replication/initialization upto point to run one command
  • hard to know when the slave is out of sync (working but broken) diagnose
  • does not manage binary logs for you (max-bin-logs )
  • no row level replication (5.1 row based replication, change based replication, good and bad )
  • Serialized execution on the slave
  • Master does not keep track of the slaves (to the master, the slave is just another connection) Jeremy comment “it really really sucks in production systems.”
  • No multi-master replication. A slave can not have more then one master.
  • ring replication. No idea when something breaks what is right.
  • no ability for delayed duplication.
  • no way to get binlogs back. (manually twink the info file)
  • Master doesn’t care what data is on the slave.
  • A replication backup is really only good for the restoration of that machine
  • Default reconnection timeout is way, way to high (default of 1 minute). It should be at most 1 second with exponential fallback. (master-connect-retry), no fallback, no max number of retries, logged in error log every time.

Feature Results (Things replication needs, what you want to see)

  • Delayed Replication
  • Registered Salves in the Master
  • Import Binlog
  • Checksum Table Events (Need ability for table checksum to be added to binlog periodically so it can be checked by the slave.)
  • Global Sequence Number
  • Connect Retry Exponential Backoff
  • Hetrogous Replication (Oracle to MySQL). Golden Gate Software has a commercial offering
  • Command Exclusion List (sql_log_bin=0)
  • Replication filters by data on the slave
  • Show upcoming queries, skip query
  • Multi-Master to one slave
  • piping mysqlbin log commands into the mysql client fails for some character sets.
  • Binlog index capability
  • Checksum of Binary Events to determine a command is valid
  • command line interface in mysqllog so you could go backwards and forwards, then execute commands.

Check out more at Google Code Blog.

Day 2 – Memorable Quotes

by

Continuing on from my Day 1 – Memorable Quotes from the MySQL Camp.

“Are there any big replication users” — Jeremy “Are you calling me fat again” — Sheeri

“Only some of us have problems with interruptions.” — Jeremy to Jay

“It really really sucks in production systems.” — Jeremy About Slave management by Master.

“So there are like 12 people here, it must be the CEO’s turn to talk.” — Marten Mickos MySQL CEO

“Kegs and Eggs” — Joel S. Regarding all beer that will still be available at breakfast tomorrow.

“You can fight to the death for it”, Jeremy to his two employees Joel and Justin about who gets to be called employee #1.

“Patches go to employee #1″ — Ronald directed to Joel when a replication patch was coined by Jeremy and Eric.

“It’s a little like Google, there are no numbers”. In response to getting any dates/times on a commitment to functionality by MySQL.

“There is a way, but you don’t want to do it.” — Monty on a topic in using Replication Slave for Master Backups

“The Blackhole storage engine is really really scary. It’s not just the name, it’s a hack.” — Jeremy

“It will suck you in.” — More on the Blackhole Storage Engine.

“It still scares me.” — Jeremy are a long discussion by Brian on the Blackhole architecture concluding with the transactional state.

“I’m not sure I’d buy that.” — Brian continuing on more comments about the blackhole discussion.

“Let’s not optimize things that won’t happen in the grand scheme of things” — Jeremy

“You asked what I wanted to see, not what was practical” — Sheeri

“Wasting network bandwidth is great” — Jeremy

“People do lots of weird things to do performance”.

“All you need is beer and love”.

“Oh” — Sheeri. Long pause. “Light bulb pops up” — Jeremy

“Are we eating Oracle’s lunch? No we are eating Oracle’s dessert” — Marten Mikos MySQL CEO

MySQL Winter of Code

by

Our first session in Day 2 of the MySQL Camp was the MySQL Winter of Code, as well as an overview of the QA Pilot program and Overview of the Community Doxygen Project by Kaj Arnö and Jay Pipes.

Starting with discussions on Code Contributions & MySQL Winter of Code

Quality Contributer Program

  • More coding happens during wintertime then in summer
  • MySQL has less contributions than many other Open Source projects
  • Contributor License Agreement
    • We want to award contribution more then nominally
    • We want to encourage contributions in all areas
    • We prefer contributions in certain areas (especially encourage them)

Requirements for Winter of Code

  • A signed Contributor License Agreement
  • A well-formed proposal
  • Votes from the Community and/or MySQL

Topics for Winter of Code 2007

  • Connectors
    • Improvements in (pure drivers for) Perl, Apache APR, Python, Ruby
  • Storage Engines
    • File System Storage Engine
      • select directory,filename,size from files where size > 1000000;
      • select directory,sum(size) from files group by directory;
    • JPG/EXIF Storage Engine
      • update jpgfiles SET Author = ‘name';
  • Anything
    • Full Text Search for CJK
    • MySQL GIS improvements
    • Your Idea

Which versions does it go to?

  • MySQL 5.1 Community Server
  • MySQL 5.2 Enterprise Server

MySQL Quality Contributor Program

  • Searching for Quality Contributors
    • Bug Reports
    • Test Cases
    • Bug Patches
  • Defining a Quality Contributor
  • Encouraging Quality Contributors
    • Fixing Bugs
    • Responsiveness and feedback
    • Recognition and attribution
    • Privileges/Awards

Day 1 – Memorable Quotes

by

Plenty of people are writing highly technical stuff from MySQL Camp including your’s truly. However there needs to be a lighter side here, and well this is it, Memorable Quotes.

“That’s moderately easy to difficult.” Brian Aker talking about table_funcs in A MySQL Core Kernel

“That’s Trivial, it’s less then a day’s work”, Monty, also in “A MySQL Core Kernel”, of course Monty said “It’s Trivial” several times, and that’s fine, it probably is trivial and is a day’s work for the guru’s, the problem is there are presently 6,000 trivial day’s work on the list of things to do.

“I’m trying to estimate when my finger will fall off.” — Jay Pipes You had to be there. I will say no more.

“You work for InnoDB, right” — Dathan Vance Pattishall of Flickr “InnoDB works for me.” — Ken Jacobs of Oracle

“Absolutely” Steve Gunn of Google in “The MySQL at The Google” talk. And the question from the floor that prompted this response “Do schema changes ever affect the production systems”.

“Everything at Google grows at the rate Google grows. If you want a proper answer we have to file that with the SEC”. Steve Gunn of Google again in “The MySQL at The Google”.

“We like to use boxes that crash.” — Mark Callaghan of Google.

“I want to make it, but we have already met before.” — Paul Tuckfield while Jeremy bashing. Side Note, apparently I’ve been saying “bagging Jeremy” which is Aussie Slang, but here in the US it has other meanings!

“I’d love my business card to say Hacker Herder”. The very cool Leslie, our Google Liason person.

“Actually they are just extras, they have all been hired for the day.” — Sheeri. In reference to all the Google Employees wearing Google shirts.

“And we’ll give you a tee-shirt” — An Google employee about Job Opportunities.

“I’m going have to kill Jeremy. This wireless stinks, I’d rather have dialup” — Sheeri about our hotel connectivity, hotel being recommended by Jeremy.

“I’m the former founder of Live Journal.” — Brad Fitzpatrick. “How can you be a former founder” — Jeremy Cole.

There were of course so many more, I just didn’t write them down. But tomorrow I will be prepared.

Testing on the toilet

by


Yes you got it, even while in the restroom here at Google (you can’t say toilets here in the US, because that’s the device), Google keeps you occupied while standing or sitting with the writings of “Testing on the Toilet”.

In Episode 19, TOTT talks about “Converting Old Style Tests”. An interesting read, rather then the daily grind of the front page of USA Today, plus as well as something that can be obviously changed at a longer frequency.

So how was the toilet experience here at MySQL Camp. Well you have toilet warming seats , my first experience, it was a little weird, and then you get the builtin “bidet” as well, with the ability for front cleaning, rear cleaning and then drying. Now that was really weird.

There has been a policy of what photos we can and can’t take and that’s cool, so I can’t post a copy of it. I will however show you this cool testing logo of TOTT as it was also on a tee-shirt (yes, it’s a little stained, but geeks do that sometimes) of a Google employee in Kiev, which is where we can take photos.

MySQL Camp – Introductions & Comments

by

The great thing about this unconference, is the lack of total formal structure. For now , our first session we are having an open introduction of people, there are at good 60+ people here already, and people rolling in, and it’s great to hear people’s background, and also to bag Jeremy Cole at every opportunity. We have a variety of people from various backgrounds, companies and experience levels.

We are in the Kiev room, with power build into the desks, lots of desk space and full 360% swivel chairs. This is just another example of the company’s clear thinking about it’s requirements.

There have already been some very funny stories, I should have made more earlier notes. Here are some.

Adam Ritter (Proven Scaling ride winner) was the first to bag Oracle, really bold move with Ken Jacobs from Oracle directly behind him, and he had already made his introduction.

Paul Tuckfield of You Tube guys said to Jeremy Cole re his replication talk “I want to make it, but we have already met before.” There have been about 10 bags of Jeremy already, he is giving as good as he is getting. Proven Scaling are sponsoring Beer session tomorrow night. Great stuff Jeremy. He did also ask how many people were planning on coming, given the number of people at the MySQL Camp has tripled in the past few days.

Breaking news. Mark Callaghan from Google, “Is there anybody from You Tube here”, to which the Paul Tuckfield of You Tube identified himself. After a few quick words the Google comment was “Deals Off” which made everybody laugh. That’s been level of good interaction with people. here

Flickr DB dude (his words) Dathan Vance Pattishall said to Ken Jacobs “You work for InnoDB, right”. Ken Jacobs response was “InnoDB works for me.”. Again a lot of laughs.

Google update – another 2 mins later

by


I’m outside enjoying a very lovely Danish and Orange juice with Jay and Leslie, and like 3 motorised scooters and a guy on a skateboard goes past. Did I mention how cool this place is!

Back in the foyer and Sheeri is sitting in the leather massage chair, as more people start streaming in. She has her laptop there and is IM’ing her boyfriend.

“So I’m in a massage chair at Google head quarters”. And his response is , “like right now”. Well here will be the photo and video when we find somebody with a card reader for my camera.

Jay’s looking a little worried, registrations are now over 200, 202, yesterday is was 150, and like 3 days ago, still in the 70s & 80s. People must have found out a free event at MySQL. We are going to kick people out that don’t contribute. It is a unconference.

So now that I write this, registrations are at 206.

My own Googlewear

by

So like two minutes later, some official looking Google people come over and saw “Come on over and get your Google Shirt”. So before the last past is even cold, we have our own Googlewear.

A minute later, Leslie is back again saying, guys and lady (Just for Sheeri), “Contintental breakfast is ready in the room”. Now to check out the Google Food!

Googlewear

by

Everybody here (that is not us visitors) are wearing Google shirts. It must be an official clothing label.

So Sheeri says “Actually they are just extras, they have been hired for the day.”

So the latest quote from Leslie is “Eat, joy and be merry, and stay inside the blue lines”. Of course I should also mention when we arrived the parking security guy said. “Follow the second yellow brick road”. This is going to be a weekend just of quotes!

MySQL Quotes

by

Frank was on a role with MySQL quotes (it’s 1am here in New York – All that Red Bull & Vodka). Here are some of them:

Let me scale you!

Wanna scale.

Scale me Baby!

Backup Now!

MySQL – DBA Friendly.

MySQL – Use the Attitude.

MySQL. Be Bold!

MySQL. Look Again

MySQL – Coming to a website near you.

One small step for Data, one giant leap for DBA.

Data, we are serious about it.

My Job, My Passion. MySQL.

MySQL. Never Doubt.

MySQL. Scaling made Easy.

MySQL. Scaling all you want.

Got MySQL!

Do it with MySQL.

Scale Yourself.

MySQL or die.

I’ve also done some of my own shirts designs (see small images below), a number I already have on shirts (you can check them out on me at MySQL Camp).

Some other references include MySQL forge Merchandise and Arjen’s suggestions.


UltimateLAMP Passwords

by

For those passwords I have not clearly mentioned within the included wiki documents, here is a summary.

  • UltimateLAMP is built on the VMWare Browser Appliance application, so all Linux passwords match the supplied default.
    • vmware/vwmare
    • root/vmware To use, you do $ sudo su –
  • For all UltimateLAMP passwords, I have used dolphin. Within MySQL, the username will match the database name of the application.
  • For all UltimateLAMP Applications, I have used a username that matches the product name (where possible), and have used the password dolphin. Some applications for example, have a default user (for example WordPress uses admin).

A summary introduction to Agile

by

Agile Development Methodology: – Most popular Implementations: Extreme Programming (XP), SCRUM, Crystal

Links

Books Highly Recommended

Extreme Programming Explained Extreme Programming Pocket Book More books on my Library page.

Linux One Liner – dirtree alternative

by

Linux has a cool command called dirtree that gives a more visual representation of your directory structure. If you have the misfortune of working on a Unix variant that doesn’t have it, checkout this cool one liner.

ls -R . | grep ":$" | sed -e 's/:$//' -e 's/[^-][^/]*\//--/g' -e 's/^/   /' -e 's/-/|/'

Thanks for the command Tom.

Log Buffer #13: a Carnival of the Vanities for DBAs

by

Unlike fellow author Giuseppe of last week’s Log Buffer #12 I volunteered for the job of this week’s Log Buffer. Lots to say, so little time, so lets get started with Log Buffer #13.

Tom Kyte has been at the DBForum 2006 in Denmark. Apart from the contents of the Forum, his picture and comment “I spied some artifacts from Mogens Oracle Museum, a copy of the Version 3 and Version 4 Oracle” in Dbforum 2006, in the past… was an impressive look back in time. Manuals, what are they? So how old is this? Wikipedia History places Oracle Version 4 at 1984, some 22 years ago. One of the comments to Tom’s entry takes us to Back to the future (Oracle 4.1 VM appliance). The title gives the article’s content away, but worth a view of Oracle history. Good to also see Tom won the 42 Question Quiz on day 1, but what was the question he got wrong?

Ric Smith gives us a window of this month’s upcoming Oracle Open World with some details of Oracle Open World 2006 – Oracle Develop, “a new event tailored for the “geek” in us all. The format will make for a more developer-oriented conference”. Craig Mullins is at the European International DB2 User Group conference being held in Vienna this week (must be the month for RDBMS conferences). Details of his presentation “Change Control for DB2 Access Paths” are at IDUG in Vienna.

Build Your Own Oracle RAC Cluster on Linux – Again references a very detailed article and explanation by Jeffrey Hunter on RAC and shows the benefit by contributions to the OTN Oracle Community. If you’re heading to Oracle Open World this month, Justin Kestelyn mentions details of a similar presentation being held during the “Oracle on Linux Experience” portion of OTN Night on Monday Oct 23.

David Aldridge in his article Linux 2.6 Kernel I/O Schedulers for Oracle Data Warehousing: Part II has received some good responses in his concise and simple Benchmark. I always strive for simplicity in solving problems and this looks like a good simple approach to graphing I/O.

A little off the beaten track is Applying Web 2.0 to the Enterprise by Jonathan Bruce. The reason why I mention this is two fold. Firstly, decisions made by Project Management can have a big effect on the software development process, and this can have a significant effect on the DBAs and System Administrators that support systems. The article also mentions Agile Software Development of which I am a strong proponent. As I have a very detailed database background I’m also wary of some of the “strenghts” mentioned generally with Agile. A topic I’m happy to discuss more at some time.

E A D G B E. I have no idea what that means, you will need to read Ian Thain’s article regarding the Sybase WorkSpace to find out. He publishes some healthy performance improvement throughput figures with his 3 tuning guidelines.

Firebird 2.00 Release Candidate 5 has also been released this week. The news article indicates that this will probably become the release version.

Peter Scott reminds us that with all the technology advances and an existing 8 year old system which includes documentation, things still happen in I hate on call.

Greg Sabino Mullane over at Planet PostgreSQL is keeping abreast of the various open source offerings with his report on Berkeley DB now does MVCC. His comment “Looks like Oracle is actually doing something with their purchase …. Curiously, this comes right at the point when MySQL is dropping the BDB engine from their product.”. Hmmm, interesting observation, however it wasn’t the actual reason why BDB was dropped from MySQL. The actual reason mentioned some time ago can be found at BDB Engine removal.

Are we working in a booming industry? “Overall, Gartner is predicting that the worldwide DBMS market is around $14 billion and will continue to grow by nearly 7% per year”. This comment by Zack Urlocker is from his attendance of the “Gartner Open Source Summit - a very thorough analysis of the impact of open source technology in the database market.” You can read all his comments of the summit in Gartner Mastermind panel and Gartner on Open Source Databases.

Exploring the secrets of intermediate materialization by Adam Machanic revives a trick he had in SQL Server 2000 in improving logical reads when query tuning. This example shows it’s operation in SQL Server 2005.

Peter Zaitsev gives us a quick refresher on his MySQL Performance Blog with What to tune in MySQL Server after installation. A good introduction reference of configurable system variables, particularly for those non-MySQL DBA’s that need to also support a MySQL installation. Mike Kruckenberg also gives us a valuable consolidated reference in his twin articles, Guide to Incompatibilities when Upgrading MySQL to Version 4.1 and Guide to Incompatibilities when Upgrading MySQL to Version 5.0. Essential reading for clearly understanding MySQL database upgrades and possible traps.

MySQL Tools for Microsoft Visual Studio 1.0.1 beta has been released. Enough said. Ok, well for those that want some more detail, I quote from Reggie Burnett – a downloadable plug-in for Visual Studio 2005 that allows Windows developers to quickly build MySQL data-driven applications with Visual Studio. With this plug-in, developers will be able to create, modify and manage MySQL database objects with an easy-to-use interface from within the Visual Studio IDE.. If only I used Microsoft I could check it out!

Daniel Schneller highlights one of the problems in a large scale out MySQL implementation in his article MySQL replication timeout trap. Valuable information in a network infrastructure to ensure your slaves are performing optimally.

Normally I’d summarise a worthy article for review, this time I’ve reproduced the concise summary by Jason Gaylord in Preventing SQL Injection Attacks which explains his content. Scott Guthrie just posted some really good stuff about preventing SQL injection attacks. In his blog post he talks about an application that Michael Sutton created to check SQL injection attacks by screening Google search and looking for sites with QueryString, etc. Check out his post for more details: http://weblogs.asp.net/scottgu/archive/2006/09/30/tip_2f00_trick_3a00_-guard-against-sql-injection-attacks.aspx

Of the big 5 or 6 RDBMS products of the past 2 decades, DB2 is the only one that hasn’t crossed my path in some way. Willie Favero writes What’s in a name – The saga continues…, sharing his views on the official name of DB2® Version 9 for z/OS.

Jeremy Cole has been busy in recent months with his new found freedoms in his new venture Proven Scaling. He has released another MySQL Source Patch with On Triggers, Stored Procedures, and Call Stacks. Keep em comin’ Jeremy. And just as I complete this weeks Log Buffer, good mate Jay (the plumber) Pipes has published HOWTO: Making a Corresponding Test Case for your Patch. Very worthy information for all those past, present and future patch writers.

The Data Charmer Giuseppe Maxia gives us the inside goss on his recent vacation interests in Take the MySQL Certification in five steps. Good Advice, I liked Point 5, and the unofficial Point 6. Marcus Popp also points us to New Lists of Certified Candidates online so you can see your name in lights. Reminds me to stop procrastinating and to take the MySQL 5 exams myself. It’s been on the cards for a few months now.

We end this week with one of those feel good stories of something that inspires me. Paul McCullagh has written his own MySQL transactional storage engine. PBXT beta 0.97 has just been released as a Pluggable storage engine for MySQL 5.1. Quoting Paul “PBXT is the first full featured engine to be released in this form.”. This leverages a new feature in the upcoming MySQL 5.1 GA release where developers can use MySQL’s extensible Storage Engine Architecture as a plugin without the need for recompiling with MySQL source. Look out for a lot more opportunities in storing and access different types of data in the future with this feature. [Author Side Note: Compiling MySQL from the latest BK tree may contain code features that are not fully tested (e.g. 2 Oct 2006). It’s best when integrating other patches or plugins to use a known MySQL Source Snapshots, otherwise things may break!]

And with a certain amount of deja vu from last week’s closing Log Buffer #12 comment by Giuseppe, your’s truly will also be joining MySQL. Checkout my If you can’t beat them, join them.

That’s all for lucky #13. Thanks for the opportunity Dave.

If you can't beat them, join them!

by

Like fellow friends and MySQL’ers before me Morgan, Roland, Giuseppe, Markus and Sean, I’ve joined the MySQL juggernaut on the ride of my life, achieving two of my short/medium term professional goals in one step. Woot!

It says something to me about the company I’m very excited to work for when I knew of all these people before they joined MySQL this year (2006). I’ll also be joining other friends and MySQL people Arjen, Jon, Jay, Colin, Michael Z and I still have a list of friends that I’ve met while being part of the MySQL community.

And as Giuseppe said I’ll be working in a virtual company. Another article I like to tell others about MySQL is MySQL: Workers in 25 countries with no HQ.

I’ll leave you with the MySQL Values from the Company About MySQL AB page.

We want the MySQL server to be:

  • The best and the most used database in the world
  • Available and affordable for all
  • Easy to use
  • Continuously improved while remaining fast and safe
  • Fun to use and improve
  • Free from bugs

MySQL AB and the people of MySQL AB:

  • Subscribe to the Open Source philosophy
  • Aim to be good citizens
  • Prefer partners that share our values and mindset
  • Answer email and give support
  • Are a virtual company, networking with others

Tutorial – Beginner Web Services

by

An introduction to using Axis.

What is Axis?

Axis is essentially a SOAP engine — a framework for constructing SOAP processors such as clients, servers, gateways, etc. The current version of Axis is written in Java. But Axis isn’t just a SOAP engine — it also includes:

  • a simple stand-alone server,
  • a server which plugs into servlet engines such as Tomcat,
  • extensive support for the Web Service Description Language (WSDL),
  • emitter tooling that generates Java classes from WSDL.
  • some sample programs, and
  • a tool for monitoring TCP/IP packets.

Pre-Requisites

Installation

su -
cd /opt
wget http://apache.ausgamers.com/ws/axis/1_4/axis-bin-1_4.tar.gz
tar xvfz axis-bin-1_4.tar.gz
ln -s axis-1_4/ axis
echo "AXIS_HOME=/opt/axis;export AXIS_HOME" > /etc/profile.d/axis.sh
. /etc/profile.d/axis.sh
cp -r $AXIS_HOME/webapps/axis $CATALINA_HOME/webapps
catalina.sh stop
catalina.sh start

At this time, you should be able to confirm this installation was initially successful by going to http://localhost:8080/axis/

Installed Axis Options

The default Axis page, gives you a number of options. To confirm the installation, select the Validate Axis Link http://localhost:8080/axis/happyaxis.jsp. If there is anything missing this page will report it. In my case I was missing XML Security, which is optional.

cd /tmp
wget http://xml.apache.org/security/dist/java-library/xml-security-bin-1_3_0.zip
unzip  xml-security-bin-1_3_0.zip
cp xml-security-1_3_0/libs/xmlsec-1.3.0.jar /opt/tomcat/common/lib
catalina.sh stop
catalina.sh start

One of the links from the default home page are http://localhost:8080/axis/servlet/AxisServlet which Lists services.

First Use

One of the nicest parts of AXIS is its “instant Web service” feature called Java Web Service (JWS) — just take a Java file, rename it, and drop it into TOMCAT_HOME/webapps/axis to make all of the (public) methods in the class callable through Web services.

Quote.java

import java.util.HashMap;
import java.util.Map;

public class Quote {
  private HashMap quotes = null;
  public Quote() {
    quotes = new HashMap();
    quotes.put("Groucho Marx", "Time flies like an arrow.  Fruit flies like a banana.");
    quotes.put("Mae West", "When women go wrong, men go right after them.");
    quotes.put("Mark Twain", "Go to Heaven for the climate, Hell for the company.");
    quotes.put("Thomas Edison", "Genius is 1% inspiration, 99% perspiration.");
  }
  public String quote(String name) {
    String quote;
    if (name == null || name.length() == 0
      || (quote = (String) quotes.get(name)) == null) {
      quote = "No quotes.";
  }
  return (quote);
  }
  public int count() {
    return quotes.size();
  }
}

cp Quote.java /opt/tomcat/webapps/axis/Quote.jws

http://localhost:8080/axis/Quote.jws
http://localhost:8080/axis/Quote.jws?wsdl

More details can be found at Getting Started using Web Services with Tomcat and Axis.

What’s Next

In my next Tutorial, I’ll be moving to the practical use of Web Services using WSDL.

References

When is a batch job successful?

by

Simple enough question, and it’s a simple enough answer. When the batch job/process in question successfully completes what it is designed to do and not in error.

I’m attempting to test, integrate and document some developed code on a client site, and well, I’m disgusted. (as with most things, is an accumulation of a number of things that lead to these frustrations.)

The process is broken down into two parts, lets call these X and Y. Now Y is the most stable part of a long standing product, it’s the API calls to the database. X does some pre-processing, then calls Y, then reports back success/failure.

Simple enough, and these are batch processes run after hours, so operators that don’t have the business knowledge need to know success or failure.

I’ll set aside for the moment that the calling process (which is indeed a shell script wrapper around the Java code) returns a status 0-Success and 1-Failure. This is practically useless because even when X fails, it doesn’t necessary report that (another story, but part of the same frustration)

I’ve extracted a small portion of the XML response that is returned from Y, that is then inteperated by X.

...
 <Status>
  <Code>0</Code>
  <Description>Success</Description>
  <DateTime>2006-09-26T16:03:45</DateTime>
 </Status>
 <Result>
   <OutputFileName>Not.Real.Name.Output.txt</OutputFileName>
 </Result>
...

...
<Severity>
  <Code>FATAL</Code>
    <Description>A fatal error was encountered while processing. See the reason code and description for further details.</Description>
</Severity>
<Reason>
 <Code>50</Code>
 <Description>XML exception: XML parse error on line: 9, position: ....
</Reason>
...

I’m not being told, “Oh, that’s a problem”. I’m been attempted to be convinced that it’s not an error, it is success.

Well, I don’t know from what planet you have lobbed in from, but in by book, FATAL is FATAL. Check out Handling Error Levels in Logging.

What’s the most depressing is I’m expected to hand this over to the customer for testing. My job isn’t actually testing, it’s integration and documentation for the end user, but the level of quality has demanded that I test it onsite before passing on. Well, I’m not going to give this to the customer, which makes it hard when the developers (who are on the same team as me) don’t see this as a problem.

PS: The list of articles of this nature has grown to the point, I’ve created my own “The Daily WFT” category. I’ve had a lot of stories I’ve never written about, perhaps I’ll pen a few more now.