The case against using rpm packaging for MySQL

In some environments using a distro package management system may* provide benefits including handling dependencies and providing a simpler approach when there are no dedicated DBA or SA resources.

However, the incorrect use can result in pain and in this instance production downtime. Even with dedicated resources at an unnamed premium managed hosting provider, the simple mistake of assumption resulted in over 30 minutes of unplanned downtime during peak time.

One of the disadvantages of using a system such as rpm is the lack of control in managing the starting and stopping of your MySQL instance, and the second is unanticipated package dependency upgrades.

So what happened with this client. When attempting to use the MySQL client on the production server, I got the following error.

$ mysql -uxxx -p
error while loading shared libraries: libmysqlclient.so.10: cannot open shared object file: No such file or directory

The server was running MySQL 5.0.27 via an rpm install.

$ rpm -qa | grep -i mysql
MySQL-server-standard-5.0.27-0.rhel3
MySQL-shared-standard-5.0.27-0.rhel3
MySQL-devel-standard-5.0.27-0.rhel3
MySQL-shared-compat-5.0.27-0.rhel3
MySQL-client-standard-5.0.27-0.rhel3

With no access to this managed server the information was relayed to the hosting provider and some time later we found the production website down. Some 30 minutes later we found that to fix the rpm problem, a dependency upgrade has also caused an automatic upgrade from 5.0.27 to 5.0.88.

While upgrading is not necessarily a bad thing, the lack of planning including a backup, a scheduled window of downtime and any level of testing is simply a poor cowboy approach to DBA management.

Upcoming MySQL Conferences

Unlike previous years when the number of conferences with MySQL content diminishes after the O’Reilly MySQL and OSCON conferences (Open SQL Camp excluded), this year has a lot on offer.

This month:

Upcoming next month in September:

  • MySQL Sunday at Oracle Open World on September 18 in San Francisco includes 4 tracks and around 15 quality speakers. (Big numbers of attendees also rumored but yet unconfirmed).
  • The inaugural Surge Scalability conference in Baltimore will include presentations by myself and Baron Schwartz (Percona being sponsors) as well as talks from other popular sites using MySQL.

If your in SF for the MySQL Sunday you may also want to come for the SF MySQL Meetup on the preceeding Thursday night where I’ll be giving my talk on “Common MySQL Scalability problems, and how to fix them”.

In October:

  • Open SQL Camp in Boston from Friday, Oct 15th in the evening, ending Sunday Oct 17th

Europeans will be busy in November where you will find dedicated MySQL tracks with multiple speakers at DOAG and UKOUG. Other MySQL talks can be found at SAPO Codebits 2010 and BGOUG.

And for South America, stay tuned. October will be your month!

There is also a great event calendar maintained by the MySQL community team on the Forge.

Why GRANT ALL is bad

A common observation for many LAMP stack products is the use of poor MySQL security practices. Even for more established products such as WordPress don’t always assume that the provided documentation does what it best for you. As per my earlier posts where I detailed installation instructions and optimal permissions for both WordPress and Mediawiki, and not just directed readers to online documentation.

In this post I will detail why GRANT ALL is bad.

Let’s start with what GRANT ALL [PRIVILEGES] provides. As per the MySQL 5.1 Reference Manual you get the following:

ALTER, ALTER ROUTINE, CREATE, CREATE ROUTINE, CREATE TEMPORARY TABLES, CREATE USER, CREATE VIEW, DELETE, DROP, EVENT, EXECUTE, FILE, INDEX, INSERT, LOCK TABLES, PROCESS, REFERENCES, RELOAD, REPLICATION CLIENT, REPLICATION SLAVE, SELECT, SHOW DATABASES, SHOW VIEW, SHUTDOWN, SUPER, TRIGGER, UPDATE, USAGE

I am going to focus on just one privilege that is included with ALL, and that is SUPER. This privilege can do the following which can be destructive for an application level user:

  • Bypasses read_only
  • Bypasses init_connect
  • Can Disable binary logging
  • Change configuration dynamically
  • No reserved connection

User Permissions

This is how a user should be created, granting only the required permissions to a given schema.

CREATE USER goodguy@localhost IDENTIFIED BY 'sakila';
GRANT CREATE,SELECT,INSERT,UPDATE,DELETE ON odtug.* TO goodguy@localhost;

This is what is commonly seen.

CREATE USER superman@'%';
GRANT ALL ON *.* TO superman@'%';

Bypasses read_only

Many MySQL replication environments rely on ensuring the MySQL slave is consistent with the master. Did you know that an application can bypass this security when read_only=true is used?

$ mysql -ugoodguy -psakila odtug
mysql> insert into test1(id) values(1);
ERROR 1290 (HY000): The MySQL server is running with the --read-only option so it cannot execute this statement
$ mysql -usuperman odtug
mysql> insert into test1(id) values(1);
Query OK, 1 row affected (0.01 sec)

GRANT ALL is bad for data consistency.

Bybasses init_connect

A common practices used for UTF8 communications is to use the init_connect configuration variable.

#my.cnf
[client]
init_connect=SET NAMES utf8
$ mysql -ugoodguy -psakila odtug

mysql> SHOW SESSION VARIABLES LIKE 'ch%';
+--------------------------+----------+
| Variable_name            | Value    |
+--------------------------+----------+
| character_set_client     | utf8     |
| character_set_connection | utf8     |
| character_set_database   | latin1   |
| character_set_filesystem | binary   |
| character_set_results    | utf8     |
| character_set_server     | latin1   |
| character_set_system     | utf8     |
+--------------------------+----------+
$ mysql -usuperman odtug

mysql> SHOW SESSION VARIABLES LIKE 'character%';
+--------------------------+----------+
| Variable_name            | Value    |
+--------------------------+----------+
| character_set_client     | latin1   |
| character_set_connection | latin1   |
| character_set_database   | latin1   |
| character_set_filesystem | binary   |
| character_set_results    | latin1   |
| character_set_server     | latin1   |
| character_set_system     | utf8     |
+--------------------------+----------+

GRANT ALL is bad for data integrity.

Disables Binary Logging.

$ mysql -usuperman odtug

mysql> SHOW MASTER STATUS;
+-------------------+----------+--------------+------------------+
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| binary-log.000001 |      354 |              |                  |
+-------------------+----------+--------------+------------------+

mysql> DROP TABLE time_zone_leap_second;
mysql> SET SQL_LOG_BIN=0;
mysql> DROP TABLE time_zone_name;
mysql> SET SQL_LOG_BIN=1;
mysql> DROP TABLE time_zone_transition;
mysql> SHOW MASTER STATUS;
+-------------------+----------+--------------+------------------+
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| binary-log.000001 |      674 |              |                  |
+-------------------+----------+--------------+------------------+
$ mysqlbinlog binary-log.000001 --start-position=354 --stop-position=674

# at 354
#100604 18:00:08 server id 1  end_log_pos 450 Query thread_id=1 exec_time=0 error_code=0
use mysql/*!*/;
SET TIMESTAMP=1275688808/*!*/;
DROP TABLE time_zone_leap_second
/*!*/;
# at 579
#100604 18:04:31 server id 1  end_log_pos 674 Query thread_id=2 exec_time=0 error_code=0
use mysql/*!*/;
SET TIMESTAMP=1275689071/*!*/;
DROP TABLE time_zone_transition
/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;

Should that statement be run on MySQL Slaves?
Is the binary log used for any level of auditing?

GRANT ALL is bad for slave consistency.

The reserved connection

MySQL reserved one connection for an administrator to be able to login to a server. For example.

$ mysql -uroot

mysql> show global variables like 'max_connections';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 3     |
+-----------------+-------+
1 row in set (0.07 sec)

mysql> show global status like 'threads_connected';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| Threads_connected | 4     |
+-------------------+-------+

mysql> SHOW PROCESSLIST;
+----+------+-----------+-------+---------+------+------------+---------------
| Id | User | Host      | db    | Command | Time | State      | Info
+----+------+-----------+-------+---------+------+------------+---------------
| 13 | good | localhost | odtug | Query   |  144 | User sleep | UPDATE test1 ...
| 14 | good | localhost | odtug | Query   |  116 | Locked     | select * from test1
| 15 | good | localhost | odtug | Query   |   89 | Locked     | select * from test1
| 15 | root | localhost | odtug | Query   |   89 | Locked     | SHOW PROCESSLIST

However if all application users are already using the SUPER privilege, the administrator will get.

$ mysql -uroot
ERROR 1040 (HY000): Too many connections

There is no way to be able to login and see what’s happening, or kill threads for example. In this case you either wait, or you are required to kill the mysqld process, or fine the client threads to kill. The result of the former may lead to a corrupt database requiring additional recovery.

GRANT ALL is bad for system administration and monitoring.

Conclusion

Don’t use GRANT ALL for application users. For more information, including why I only listed just 5 issues, check out my MySQL Idiosyncrasies that BITE presentation. I will also be presenting this talk at MySQL Sunday at Oracle Open World in September.

You have to love the Planet MySQL voting system

Within a few hours my post Installing Mediawiki on Oracle Enterprise Linux LAMP stack got 5 negative votes.

Wow, I’d be glad if these people could felt so passionately about all the other CRUD on Planet MySQL that has ZERO to do actually do with MySQL.

Using a LAMP product, and providing instructions for operation can’t be a negative voting offense. So it can only be the words “Oracle Enterprise Linux”.

For those negative people out there that care enough to physically mark blogs let me share some facts with you. RedHat Enterprise Linux (RHEL) is the most widely used and support platform for production MySQL environments. CentOS and Oracle Enterprise Linux (OEL) provide via the freedom of Open Source, their own offerings of RHEL with various other features including support or specific additional features for other products. Oracle also provides and commits work to the Linux Kernel, so they are just as much an allies to open source as other companies.

So are you bagging the operating system or just the word Oracle. Oracle is hear to stay, in fact those attending the MySQL Sunday event at Oracle Open World may get a welcome wake up shock. And for reference these articles are being written for a published Oracle Magazine article which meets the comfort level of the reader and introduces MySQL in a positive way. Your actions show just how much you are unwilling to embrace the larger community.

Installing Mediawiki on Oracle Enterprise Linux LAMP stack

A company wiki can be easily configured in under 10 minutes using Mediawiki the open source LAMP software that powers the top 10 website Wikipedia.

A company wiki is an ideal means for a centralized and user contributed documentation system. The following steps show you how to download, configure and get your Mediawiki site operational.

Software Pre-Requisites

Software Installation

su -
cd /tmp
wget http://download.wikimedia.org/mediawiki/1.16/mediawiki-1.16.0.tar.gz
cd /var/www/html
tar xfz /tmp/mediawiki*.tar.gz
mv mediawiki* wiki
chmod 777 wiki/config

NOTE: You should check the Mediawiki Downloads page for the latest version.

You can now visit http://localhost/wiki and you will be presented with a message of an un-configured Mediawiki environment. You can streamline the MySQL portion of this configuration with the following commands.

mysql -uroot -p -e "DROP SCHEMA IF EXISTS wikidb;CREATE SCHEMA wikidb"
mysql -uroot -p -e "CREATE USER wikiuser @localhost IDENTIFIED BY 'sakila'"
mysql -uroot -p -e "GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX on wikidb.* TO wikiuser@localhost"

You can now complete the configuration on the Configuration Page. You will need to enter the following information.

  • Wiki name = Example Wiki
  • Contact e-mail = your email address
  • Admin user password = enter password
  • DB password = sakila

After successful installation, one additional step is needed.

mv wiki/config/LocalSettings.php wiki/
chmod 400 wiki/LocalSettings.php
chmod 500 wiki/config

You can now start using your new Wiki at http://localhost/wiki. You will find documentation at the links provided on the displayed home page and also at www.mediawiki.org.

MySQL Structures

Looking at the tables that are created by the installation process:

$ mysql -uwikiuser -psakila blog

mysql> show tables;
+-------------------+
| Tables_in_wikidb  |
+-------------------+
| archive           |
| category          |
| categorylinks     |
| change_tag        |
| external_user     |
| externallinks     |
| filearchive       |
| hitcounter        |
| image             |
| imagelinks        |
| interwiki         |
| ipblocks          |
| job               |
| l10n_cache        |
| langlinks         |
| log_search        |
| logging           |
| math              |
| objectcache       |
| oldimage          |
| page              |
| page_props        |
| page_restrictions |
| pagelinks         |
| protected_titles  |
| querycache        |
| querycache_info   |
| querycachetwo     |
| recentchanges     |
| redirect          |
| revision          |
| searchindex       |
| site_stats        |
| tag_summary       |
| templatelinks     |
| text              |
| trackbacks        |
| transcache        |
| updatelog         |
| user              |
| user_groups       |
| user_newtalk      |
| user_properties   |
| valid_tag         |
| watchlist         |
+-------------------+
45 rows in set (0.00 sec)

mysql> SELECT table_name,engine,table_rows FROM information_schema.tables WHERE table_schema = 'wikidb';
+-------------------+--------+------------+
| table_name        | engine | table_rows |
+-------------------+--------+------------+
| archive           | InnoDB |          0 |
| category          | InnoDB |          0 |
| categorylinks     | InnoDB |          0 |
| change_tag        | InnoDB |          0 |
| external_user     | InnoDB |          0 |
| externallinks     | InnoDB |          0 |
| filearchive       | InnoDB |          0 |
| hitcounter        | MEMORY |          0 |
| image             | InnoDB |          0 |
| imagelinks        | InnoDB |          0 |
| interwiki         | InnoDB |         95 |
| ipblocks          | InnoDB |          0 |
| job               | InnoDB |          0 |
| l10n_cache        | InnoDB |       3686 |
| langlinks         | InnoDB |          0 |
| log_search        | InnoDB |          0 |
| logging           | InnoDB |          0 |
| math              | InnoDB |          0 |
| objectcache       | InnoDB |          2 |
| oldimage          | InnoDB |          0 |
| page              | InnoDB |          1 |
| page_props        | InnoDB |          0 |
| page_restrictions | InnoDB |          0 |
| pagelinks         | InnoDB |          0 |
| protected_titles  | InnoDB |          0 |
| querycache        | InnoDB |          0 |
| querycache_info   | InnoDB |          0 |
| querycachetwo     | InnoDB |          0 |
| recentchanges     | InnoDB |          0 |
| redirect          | InnoDB |          0 |
| revision          | InnoDB |          1 |
| searchindex       | MyISAM |          0 |
| site_stats        | InnoDB |          1 |
| tag_summary       | InnoDB |          0 |
| templatelinks     | InnoDB |          0 |
| text              | InnoDB |          1 |
| trackbacks        | InnoDB |          0 |
| transcache        | InnoDB |          0 |
| updatelog         | InnoDB |          0 |
| user              | InnoDB |          1 |
| user_groups       | InnoDB |          2 |
| user_newtalk      | InnoDB |          0 |
| user_properties   | InnoDB |          0 |
| valid_tag         | InnoDB |          0 |
| watchlist         | InnoDB |          0 |
+-------------------+--------+------------+
45 rows in set (0.01 sec)

Sort URL for this post rb42.com/oel-install-mediawiki

First thoughts of Augen Android Internet Tablet

In the last few days there has been some press of the Augen gentouch 7″ Tablet. A new cheap tablet that is running Android 2.1.

Image from http://android-devices.net

There were a few primary motivations for getting one, the first being price, at $150 I consider cheap for a small tablet, and second it runs Android, something I’m wanting to play more with. I have an original Google G1, and I also purchased one recently for my fiance.

So what are my first impressions.

  1. First it was difficult to get. It is only available at KMart it seems. My local KMart in New York City didn’t have any. Apparently they sold out very quickly with stock on Wednesday. 10 KMart stores later (via phoning them) I found just 1 more that had even received stock, and they had 1 left. Lucky last. It was only that we had a car on Saturday it was even possible to get to this store. (-1 for ease of acquisition via company distribution)
  2. KMart claimed the price was $165, but if you checked the online magazine it was $149.99. I had to prove that first to get the price via using the web on my phone. (-1 for Kmart customer service)
  3. I wanted to buy and additional memory card as I knew from online review this was available however the box (which was still unopened before purchase only mentions Expansion Memory Card Slot, it doesn’t mention the actually size format. Even reading the manual after purchase talks about a SD card or a TF card, but is not specific. I ended up purchasing a Sandisk Mobile microSDHC 16GB card. Even that packaging was confusing as it clearly states “For Mobile Phones” twice on the package. (-1 for ease of information/packaging)
  4. Turning on the first time was a pleasant surprise because the item was fully charged. (+2 there). With my MiFi I was immediately able to connect to the web while still in the car (as a passenger). (+1 for access to primary use)
  5. The touch screen is clearly not as responsive as an Apple or smart phone, it’s mixed sometimes it only requires a light touch, sometimes a heavy touch. Early reports mentioned this, so it was not unexpected. Again for the price and proposed uses I have for it, it wasn’t a deal breaker. I saw one online video that included a stylus, and another mention online, however the manual gives no indication, and I don’t seem to have one.
  6. Downloading the 2012 HD trailer via YouTube worked promptly and without issues and picture was good (+1) but there is no external controls for audio. (-1 for that).
  7. There is a headphone jack however I find out later that this is a 2.5mm jack, and standard headphones used by everybody on the planet is 3.5mm. (-1 for that). The website claims they will supply people at no cost a set of 2.5mm headphones. Again useless as I don’t want to have to carry those around as well. What I want is a 2.5mm to 3.5mm converter, actually two because I’m sure I’ll lose it easily. (-1 for that)
  8. The main buttons for “back, menu, home” are actually on the back of the device. Once you know that it’s not that bad, but it is a little odd. Of course for right handed person. If they were on the side it’s possible any type of “death grip” may accidentally press them.
  9. Unit comes with a handy and practical leather case (+1) with adequate access for side controls, however it’s only good for holding not using because the buttons are on the back (-1).
  10. By purchasing a 16GB at $99, the price is now $250, it’s not as impressive a cost product.
  11. The purchased SDCard was not easy to install, there is no clear instruction on right side up and certainly nothing in the docs, and there are reports online that people misplace in the slot and the card gets lost inside of unit. I didn’t have that problem and I don’t really see that would happen for me. The manual indicates you will get an icon about it, however that wasn’t the case so I really didn’t know if it worked or not. (-1) Using the AndExplorer indicates a /sdcard so I assume it’s operational.
  12. There is reference to a U Disk (who knows what that is). with a supplied cable I assume it’s a means of adding a USB thumbdrive to the mini USB slot. Trying this however didn’t seem to work so I don’t know if that’s the intended use.
  13. The power supply is yet another plug I have to now carry (along with one for laptop, phone, mifi). I was really hoping the power adapter would have been a mini or micro USB. (-1)
  14. The first real use after web, video, google maps, email is to download other stuff however it seems the Android Market is broken. The official website (which I now can’t find, -1 for poor SEO and google searchability) makes a note this is broken and is expecting a patch (-2 for poor testing there.
  15. The manual was not proof read by an English person. The title cover states “Table” not “Tablet” however the funniest part was reading some the manual. I will not type what I found, but you can see the image below. It so made me laugh.
  16. There is no video output, which is really annoying because the side panel actually states HDMI in printing, but nothing physical.
  17. The screen res is 800×480. Ok, so it’s small but of the sites I’ve initially visited the horizontal bar has not been an issue. I suspect it may be in the future, but this is not my primary development machine.
  18. No webcam
  19. I wanted to download some of my ebooks, however this seems to be broken and related to the market problem. Will see after patch.

I’ve yet to really test it out, these are just my first impressions. For the price I consider it a worthwhile investment for the purposes I want it for. That is some browsing, (ideal for bedroom), I can see it a wicked 7″ GPS unit with turn by turn controls for driving, but without GPS it will be a bit manual. Reading email, or an ebook, and even use as a large digital frame, especially for my photos.

This is a gen 1 product, so you have to accept the shortcomings. You are either an alpha adopter that is willing to accept limitations and accept the benefits it does have or your not.

Other References:

Augen’s $150 Android tablet hits Kmart circular, coming to stores later this week (what first caught my eye). KMart update. First impressions of the Augen GenTouch78 Android tablet (with second mention of a stylus).