You are here: Home / Archives for Other

The minimum testing for a shared disk MySQL environment

by

Recently I was asked to provide guidelines for testing fail over of a MySQL configuration that was provided by a hosting provider.

The first observation was the client didn’t have any technical details from the hosting provider of what the moving parts were, and also didn’t have any confirmation other then I think a verbal confirmation that it had been testing.

The first rule in using hosting, never assume. Too many times I’ve seen details from a client stating for example H/W configuration, only to audit and find out otherwise. RAID is a big one, and is generally far more complex to determine. Even for companies with internal systems I’ve seen the most simple question go unanswered. Q: How do you know your RAID is fully operational? A: Somebody will tell us? It’s really amazing to investigate on site with the client to find that RAID system is running in a degraded mode due to a disk failure and nobody knew.

It took some more digging to realize the configuration in question was with Red Hat Cluster Suite. A word of warning for any clients that use this, DO NOT USE MyISAM. I’ll leave it to the readers to ask me why.

Here is a short list I provided as the minimum requirements I’d test just to ensure the configuration was operational.

Verifying a working Red Hat Cluster Suite MySQL Environment

The MySQL Environment

The database environment consists of two MySQL database servers, configured in an active/passive mode using a shared disk storage via SAN.
For the purposes of the following procedures the active server will be known as the ‘primary’ server, and the passive server will be the ‘secondary server’.
The two physical servers for the purposes of these tests will be defined as ‘alpha’ and ‘beta’, with specific H/W that does not change during these tests.

Normal Operations

Expected Configuration under normal operations.

Primary Server

  • server is pingable
  • server accepts SSH Connection
  • MySQL service is started
  • has /data appropriately mounted
  • has assigned VIP address
  • MySQL configuration file and settings are correct

Secondary Server

  • server is pingable
  • server accepts SSH Connection
  • MySQL service IS NOT started
  • DOES NOT have /data mounted
  • DOES NOT has assigned VIP address
  • MySQL configuration file is not available

1. Reboot servers ‘alpha’ and ‘beta’.

Test Status:

  • alpha server is the designated primary server
  • alpha and beta servers are operational

Action:
1.1 Restart alpha server (init 6)
1.2 Restart beta server (init 6)

Checklist:
1.3 Alpha server matches primary server configuration
1.4 Beta server matches secondary server configuration

2. Controlled fail over from ‘alpha’ to ‘beta’

Test Status:

  • alpha server is the designated primary server
  • alpha and beta servers are operational

Action:
2.1 Alpha server – Instigate Cluster failover (clusvcadm -r mysql-svc)

Checklist:
2.2 Beta server matches primary server configuration
2.3 Alpha server matches secondary server configuration

3. Controlled failover from ‘beta’ to ‘alpha’

Test Status:

  • beta server is the designated primary server
  • alpha and beta servers are operational

Action:
3.1 beta server – Instigate Cluster failover (clusvcadm -r mysql-svc)

Checklist:
3.2 Alpha server matches primary server configuration
3.3 Beta server matches secondary server configuration

Exception Operations

4. Loss of connectivity to primary server

Test Status:

  • alpha server is the designated primary server
  • beta server is online

Action:
4.1 Stop networking services on ‘alpha’ (ifdown bond0)

Checklist:
4.2 Monitoring detects and reports connectively loss
4.3 Automated failover occurs
4.4 Beta server matches primary server configuration
4.5 Alpha server matches secondary server configuration

5. Restore connectivity to secondary server

Test Status:

  • beta server is the designated primary server
  • alpha server is online, but not accessible via private IP

Action:
4.1 Start networking services on ‘alpha’ (ifup bond0)

Checklist:
5.2 Monitoring detects and reports connectively restored
5.3 No failback occurs
5.4 Beta server matches primary server configuration
5.5 Alpha server matches secondary server configuration

6. Loss of connectivity to secondary server

Test Status:

  • beta server is the designated primary server
  • alpha server is online

Action:
6.1 Stop networking services on ‘alpha’ (ifdown bond0)

Checklist:
6.2 Monitoring detects and reports connectively lost
6.3 No failback occurs
6.4 Beta server matches primary server configuration
6.5 Alpha server matches secondary server configuration

7. Restore connectivity to secondary server

Test Status:

  • beta server is the designated primary server
  • alpha server is online, but not accessible via private IP

Action:
7.1 Start networking services on ‘alpha’ (ifup bond0)

Checklist:
7.2 Monitoring detects and reports connectively restored
7.3 No failback occurs
7.4 Beta server matches primary server configuration
7.5 Alpha server matches secondary server configuration

8. Power down secondary server

Test Status:

  • beta server is the designated primary server
  • alpha server is online

Action:
8.1 Power down alpha (init 0) NOTE: Need remote boot capabilities

Checklist:
8.2 Monitoring detects and reports connectively lost
8.3 Beta server matches primary server configuration
8.4 Additional paging for extended down time for ‘degraded support for failover’

9. Loss of connectivity to primary server

Test Status:

  • beta server is the designated primary server
  • alpha server is offline

Action:
9.1 Power down beta (init 0) NOTE: Need remote boot capabilities

Checklist:
9.2 Monitoring detects and reports connectively lost
9.3 Site database connectively completely unavailable
9.4 Additional paging for loss of HA solution

10. power restored to secondary server
Test Status:

  • alpha server is offline
  • beta server is offline

Action:
10.1 Power on alpha

Checklist:
10.2 Monitoring detects and reports server up
10.3 Alpha server assumes primary role (previously it was beta)
10.4 Alpha server matching primary server configuration
10.5 Addition paging for degraded HA

11. power restored to secondary server

Test Status:

  • alpha server is primary server
  • beta server is offline

Action:
11.1 Power on beta

Checklist:
11.2 Monitoring detects and reports server up
11.3 Alpha server matching primary server configuration
11.4 Beta server matching secondary server configuration

Database Operations

12. MySQL services on primary server go offline

Test Status:

  • alpha server is the designated primary server
  • beta server is online

Action:
12.1 Stop mysql services on ‘alpha’ (/etc/init.d/mysqld stop)

Checklist:
12.2 Monitoring detects and reports database loss (while connectivity is still available)
12.3 Automated failover occurs
12.4 Beta server matches primary server configuration
12.5 Alpha server matches secondary server configuration

13. MySQL services on secondary server go offline

Test Status:

  • beta server is the designated primary server
  • alpha server is online

Action:
13.1 stop mysql services on ‘beta’ (/etc/init.d/mysqld stop)

Checklist:
13.2 Monitoring detects and reports database loss (while connectivity is still available)
13.3 Automated failover occurs
13.4 Alpha server matches primary server configuration
13.5 beta server matches secondary server configuration

14. Load Testing during failure

Test Status:

  • alpha server is the designated primary server
  • beta server is online

Action:
14.1 Agressive load testing against database server
14.2 MySQL killed without prejudice (killall -9 mysqld_safe mysql)

Checklist:
14.3 Monitoring detects and reports mysql service loss
14.4 Automated failover occurs
14.5 Beta server matches primary server configuration
14.6 Alpha server matches secondary server configuration
14.7 Beta mysql logs shows a forced MySQL Recovery in logs

15. Forced Recovery

Test Status:

  • alpha server is the designated primary server
  • beta server is online

Action:
15.1 Manual full database backup is done (in case recovery does not work). Hosting Provider not told of this.
15.2 Dummy new table/schema is created (used as verification point)
15.3 Database on alpha primary server is dropped
15.4 Hosting Provider is notified stating a full database recovery including Point In time to just before drop (no time given, only command that was run)

Checklist:
15.5 Site is marked as unavailable
15.6 Hosting Provider restore data from backup and recover to point in time
15.7 Confirmation that new table/schema is restored, and full schema is available
15.8 Site is made available
15.9 Record of time for full disaster is recorded

Conclusion

This is not an exhaustive test, in fact it is just a documented approach for consideration to show a client what the minimum testing should be. As no dry run actually occurred, there may be inaccuracies and additions necessary to this document when first executed. I would need access to an appropriate configuration in order to perform a level of testing to complete this document.

About the Author

Ronald Bradford provides Consulting and Advisory Services in Data Architecture, Performance and Scalability for MySQL Solutions. An IT industry professional for two decades with extensive database experience in MySQL, Oracle and Ingres his expertise covers data architecture, software development, migration, performance analysis and production system implementations. His knowledge from 10 years of consulting across many industry sectors, technologies and countries has provided unique insight into being able to provide solutions to problems. For more information Contact Ronald.

BIGINT v INT. Is there a big deal?

by

The answer is yes.

In this face off we have two numeric MySQL data types, both Integer. In fact MySQL has 9 different numeric data types for integer, fixed precision and floating point numbers, however we are just going to focus on two, BIGINT and INT. This design consideration is part of my recent presentation Top 20 Design Tips for Data Architects.

What is the difference?
We turn to the MySQL Reference Manual first, in 10.1.1. Overview of Numeric Types we see the following.


INT[(M)] [UNSIGNED] [ZEROFILL]

A normal-size integer. The signed range is -2147483648 to 2147483647. The unsigned range is 0 to 4294967295.

BIGINT[(M)] [UNSIGNED] [ZEROFILL]

A large integer. The signed range is -9223372036854775808 to 9223372036854775807. The unsigned range is 0 to 18446744073709551615.

Ok, well an INT can store a value to 2.1 Billion, and an a BIGINT can store a value to some larger number to 20 digits. That MySQL search didn’t help much with details, we have to dig deeper to find 10.2. Numeric Types in which we find that INT is a 4 byte integer, and a BIGINT is an 8 byte integer.

So what’s the big deal?

Quite a lot actually. Using INT rather then BIGINT can make a significant reduction in disk space. Just this one change alone can save you 10%-20% (depends on your particular situation). More significantly, when used as a primary key, and for foreign keys and indexes, reducing your index size could be 50%, and this will improve performance when these indexes are used.

My approach is this. Let’s just focus on primary keys and foreign keys to begin with. Are you going to store more then 2.1 Billion rows in your table? The answer should be no? Should you say yes, then you do have grand plans, but you are also failing to consider the ramifications of handling larger data sets (a topic for later discussion).

There are exceptions to this rule, if you do a huge number of inserts and deletes, then while you may not have 2.1 Billion rows, you may have done 2.1 Billion inserts. Again better design practices should be considered in this case.

The Test

As with everything, we need some evidence to stake the claim. Using the Sakila sample database.

We start with a simple intersection table, that has a high number of numeric only columns. This will show the best case situation.

We will create two tables, one with all BIGINT columns, and one with all INT columns and then compare the size. These tables are only small, but they show the proportion of savings of disk space.

CREATE TABLE inventory_bigint LIKE inventory;
ALTER TABLE inventory_bigint
  MODIFY inventory_id  BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
  MODIFY film_id BIGINT UNSIGNED NOT NULL,
  MODIFY store_id BIGINT UNSIGNED NOT NULL;
INSERT INTO inventory_bigint SELECT * from inventory;
CREATE TABLE inventory_int LIKE inventory;
ALTER TABLE inventory_int
  MODIFY inventory_id  INT UNSIGNED NOT NULL AUTO_INCREMENT,
  MODIFY film_id INT UNSIGNED NOT NULL,
  MODIFY store_id INT UNSIGNED NOT NULL;
INSERT INTO inventory_int SELECT * from inventory;


select table_name,engine,row_format, table_rows, avg_row_length,
        (data_length+index_length)/1024/1024 as total_mb,
         (data_length)/1024/1024 as data_mb,
         (index_length)/1024/1024 as index_mb
from information_schema.tables
where table_schema='sakila'
and   table_name LIKE 'inventory%'
order by 6 desc;
+------------------+--------+------------+------------+----------------+-------------+-------------+-------------+
| table_name       | engine | row_format | table_rows | avg_row_length | total_mb    | data_mb     | index_mb    |
+------------------+--------+------------+------------+----------------+-------------+-------------+-------------+
| inventory_bigint | InnoDB | Compact    |     293655 |             51 | 43.60937500 | 14.51562500 | 29.09375000 |
| inventory_int    | InnoDB | Compact    |     293715 |             37 | 29.54687500 | 10.51562500 | 19.03125000 |
| inventory        | InnoDB | Compact    |     293707 |             33 | 22.54687500 |  9.51562500 | 13.03125000 |
+------------------+--------+------------+------------+----------------+-------------+-------------+-------------+
3 rows in set (0.15 sec)

In this example, the data portion decreased from 14MB to 10MB or 28%, and the index portion from 29M to 19M or 34%.

CREATE TABLE customer_bigint LIKE customer;
ALTER TABLE customer_bigint
     MODIFY customer_id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
     MODIFY store_id BIGINT UNSIGNED NOT NULL,
     MODIFY address_id BIGINT UNSIGNED NOT NULL,
     MODIFY active BIGINT UNSIGNED NOT NULL;

CREATE TABLE customer_int LIKE customer;
ALTER TABLE customer_int
     MODIFY customer_id INT UNSIGNED NOT NULL AUTO_INCREMENT,
     MODIFY store_id INT UNSIGNED NOT NULL,
     MODIFY address_id INT UNSIGNED NOT NULL,
     MODIFY active INT UNSIGNED NOT NULL;

select table_name,engine,row_format, table_rows, avg_row_length,
        (data_length+index_length)/1024/1024 as total_mb,
         (data_length)/1024/1024 as data_mb,
         (index_length)/1024/1024 as index_mb
from information_schema.tables
where table_schema='sakila'
and   table_name LIKE 'customer%'
order by 6 desc;

+-----------------+--------+------------+------------+----------------+-------------+-------------+-------------+
| table_name      | engine | row_format | table_rows | avg_row_length | total_mb    | data_mb     | index_mb    |
+-----------------+--------+------------+------------+----------------+-------------+-------------+-------------+
| customer_bigint | InnoDB | Compact    |     154148 |            139 | 37.09375000 | 20.54687500 | 16.54687500 |
| customer_int    | InnoDB | Compact    |     151254 |            121 | 30.06250000 | 17.51562500 | 12.54687500 |
| customer        | InnoDB | Compact    |      37684 |            125 |  7.81250000 |  4.51562500 |  3.29687500 |
| customer_list   | NULL   | NULL       |       NULL |           NULL |        NULL |        NULL |        NULL |
+-----------------+--------+------------+------------+----------------+-------------+-------------+-------------+
4 rows in set (0.22 sec)

In this example, the data portion decreased from 20MB to 17MB or 15%, and the index portion from 16M to 12M or 25%.

NOTE: The sample data set was increased for this example.

Conclusion

Even with these simple tables and small data sets it’s clear that INT is a saving of diskspace over BIGINT. In many clients I’ve seen huge savings in multi TB databases, just with a small number of schema optimizations. If this saving alone for a more optimized database design was only 10%, it is an easy 10% that will reflect a direct improvement in performance.

About the Author

Ronald Bradford provides Consulting and Advisory Services in Data Architecture, Performance and Scalability for MySQL Solutions. An IT industry professional for two decades with extensive database experience in MySQL, Oracle and Ingres his expertise covers data architecture, software development, migration, performance analysis and production system implementations. His knowledge from 10 years of consulting across many industry sectors, technologies and countries has provided unique insight into being able to provide solutions to problems. For more information Contact Ronald.

References

Off to OSCON

by

I will be heading to my first OSCON next week where I will be presenting MySQL Proxy: from Architecture to Implementation in conjunction with Giuseppe Maxia .

As was written by Colin Charles Our booth is yours… Sun at OSCON, Sun/MySQL would appear to also have a reasonable turnout. So it will be good to see some old colleagues and friends, and hopefully meet some new contacts.

While I am based on the East Coast, I do also provide expert MySQL consulting for clients in any location. Should you like to find out more about my offerings covering Architecture, Performance, Scaling, Migration and Knowledge Transfer for MySQL Solutions, please Contact Me and I will arrange a time to meet next week.

Why SQL_MODE is important? Part I

by

MySQL pre version 5.0 was very lax in it’s management of valid data. It was easy for data integrity to be abused if you knew how. The most common examples were truncations and silent conversions that if not understood could provide a serious data integrity issue.

In version 5.0, the introduction of SQL_MODE solved this problem. We will look at one example of how SQL_MODE can be enabled to provided improved data integrity.

You want to store the individual RGB (red/green/blue) decimal values of colors in a table. Each of these has a range from 0 to 255. You read that you can store 255 values in a TINYINT Integer data type, so you create a table like:

DROP TABLE IF EXISTS color_to_decimal;
CREATE TABLE color_to_decimal(
name VARCHAR(20) NOT NULL PRIMARY KEY,
red    TINYINT NOT NULL,
green TINYINT NOT NULL,
blue   TINYINT NOT NULL);

You insert some data like:

INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('white',255,255,255);
INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('black',0,0,0);
INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('red',255,0,0);
INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('green',0,255,0);
INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('blue',0,0,255);
INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('yellow',255,255,0);

Great, but when you look at you data you get?

SELECT     name, red, green, blue
FROM       color_to_decimal
ORDER BY name;
+--------+-----+-------+------+
| name   | red | green | blue |
+--------+-----+-------+------+
| black  |   0 |     0 |    0 |
| blue   |   0 |     0 |  127 |
| green  |   0 |   127 |    0 |
| red    | 127 |     0 |    0 |
| white  | 127 |   127 |  127 |
| yellow | 127 |   127 |    0 |
+--------+-----+-------+------+
6 rows in set (0.01 sec)

What happened, you delete the data and re-insert only to find no changes. You have been the victim of a silent conversion, via a means of truncation.

The TINYINT data type is 1 byte (8 bits). 8 bits can store the values from 0 to 255. When you use this integer data type, only 7 bits are actually available, which gives the range of 0 to 127. Why? Because MySQL reserved one bit for the sign, either positive or negative, even though you didn’t want a sign.

So knowing this, you go back and recreate your table with the following definition.

DROP TABLE IF EXISTS color_to_decimal;
CREATE TABLE color_to_decimal(
name VARCHAR(20) NOT NULL PRIMARY KEY,
red    TINYINT UNSIGNED NOT NULL,
green TINYINT UNSIGNED NOT NULL,
blue   TINYINT UNSIGNED NOT NULL);

You load your data and look at it again, and you see.

+--------+-----+-------+------+
| name   | red | green | blue |
+--------+-----+-------+------+
| black  |   0 |     0 |    0 |
| blue   |   0 |     0 |  255 |
| green  |   0 |   255 |    0 |
| red    | 255 |     0 |    0 |
| white  | 255 |   255 |  255 |
| yellow | 255 |   255 |    0 |
+--------+-----+-------+------+
6 rows in set (0.00 sec)

But, should you have been told about this, should there have been an error. Well, in MySQL this is actually a warning, and most applications never support and cater for warnings. It is only when you use the MySQL client program, as in these examples, you are given an indication, with the following line after each insert. If you look closely.

mysql> INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('blue',0,0,255);
Query OK, 1 row affected, 1 warning (0.00 sec)

However there is a savior for this situation, and that is SQL_MODE.

When set to the setting TRADITIONAL, an error and not a warning is generated, and most applications support catching errors. Look at what happens in our example using the original table.

SET SQL_MODE=TRADITIONAL;
DROP TABLE IF EXISTS color_to_decimal;
CREATE TABLE color_to_decimal(
name VARCHAR(20) NOT NULL PRIMARY KEY,
red    TINYINT NOT NULL,
green TINYINT NOT NULL,
blue   TINYINT NOT NULL);
INSERT INTO color_to_decimal (name, red,green,blue) VALUES ('white',255,255,255);
ERROR 1264 (22003): Out of range value for column 'red' at row 1

As an added benefit you get this data integrity for free. We didn’t test it, because we know the data coming in is in the range of 0-255, but what if the user entered 500 for example. Let’s see.

TRUNCATE TABLE color_to_decimal;
SET SQL_MODE='';
INSERT INTO color_to_decimal (name, red, green, blue) VALUES('a bad color',500,0,0);
SELECT name, red, green, blue FROM color_to_decimal;
SET SQL_MODE=TRADITIONAL;
INSERT INTO color_to_decimal (name, red, green, blue) VALUES('a bad color',500,0,0);

Looking closely at the response in the client.

mysql> TRUNCATE TABLE color_to_decimal;
Query OK, 0 rows affected (0.02 sec)

mysql> SET SQL_MODE='';
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO color_to_decimal (name, red, green, blue) VALUES('a bad color',500,0,0);
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> SELECT name, red, green, blue FROM color_to_decimal;
+-------------+-----+-------+------+
| name        | red | green | blue |
+-------------+-----+-------+------+
| a bad color | 127 |     0 |    0 |
+-------------+-----+-------+------+
1 row in set (0.00 sec)

mysql> SET SQL_MODE=TRADITIONAL;
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO color_to_decimal (name, red, green, blue) VALUES('a bad color',500,0,0);
ERROR 1264 (22003): Out of range value for column 'red' at row 1

As discussed in my presentation Top 20 design tips for Data Architects the UNSIGNED column construct should be always defined unless there is a specific reason not to.

In some respects I would argue that the default for an Integer column should be actually UNSIGNED, and that SIGNED should be specified when you want a sign. Most integer columns generally in schema’s only contain positive numbers. There are of course plenty of examples, positional geo data, financial data, medical data for example.

One could also argue that MySQL should make the default SQL_MODE at least TRADITIONAL, and that only when you want backward compatibility should you then change the SQL_MODE.

This is Part I on SQL_MODE, there are few more interesting cases to discuss at a later time.

About the Author

Ronald Bradford provides Consulting and Advisory Services in Data Architecture, Performance and Scalability for MySQL Solutions. An IT industry professional for two decades with extensive database experience in MySQL, Oracle and Ingres his expertise covers data architecture, software development, migration, performance analysis and production system implementations. His knowledge from 10 years of consulting across many industry sectors, technologies and countries has provided unique insight into being able to provide solutions to problems. For more information Contact Ronald.

References

Sun Stock Prices

by

Sun Microsystem’s (NASDAQ:JAVA) hit a low this week of $8.71. There was a stronger rally and a close at $9.16 today. The financial times reports Sun Micro chief sees rays of hope, and Bloomberg Sun Rises After Fourth-Quarter Profit Tops Estimates.

I cashed out in March at $16.32, so that’s like a 50% drop in share price. I was lucky having been at MySQL long enough to have options to vest. Newer employees are not that lucky. I certainly hope MySQL Sun Employees get the Q4 weighted bonuses. (A structure I didn’t believe compensated with the old bonus structure).

I have been following more closely since Matt Asay’s comments in Who is buying Sun?



Image courtesy of Google Financial’s.

A Bill Gates bio

by

In the recent Wired magazine (yes, the paper one), there was an interesting time line of Bill Gates. It was rather an odd format, but I found the two page spread an enjoyable read. Some things of note in his early childhood, tips perhaps for us wanting to be successful.

  • 1968 Gates and Allen learn basic and blow entire school budget of computing time in a few weeks.
  • 1968 Employed to report PDP-10 software bugs.
  • 1971 Writes class scheduling program that places him in classes with the “right” girls.
  • 1973 Photographic memory, lucky him.
  • 1975 Writes Basic for MTIS.
  • 1976 Registers the trade name Microsoft.
  • 1980 Buys QDOS for $50k, later renaming and reselling as DOS.
  • 1984 Microsoft is one of the first software developers on Macintosh.
  • 1986 Company goes public.
  • 1996 Daily income is $30million, that’s per day.
  • 1998 The famous pie incident.

So it seems, finding bugs, using technology to meet the right women, buying and reselling somebody else’s work worked out, but will it work now.

I read elsewhere that the companies of today, such as Amazon, eBay, Google, FaceBook etc didn’t even exist 15 years ago.

There is still the opportunity for people out there, like you and I to be billionaires. Like one of my fridge magnets states. “Life isn’t about finding yourself. Life is about creating yourself.” Time to go create the next great thing we all must have.

Focus on what you do best

by

When you have a great idea for a web application, it can be hard to consider with all the moving parts to focus just on what’s your uniqueness or differentiator from everybody else.

You may want to have control over your forums, comments, chat, photo management etc, i.e. user data, but how much does that help you. Is allocating resources to these features when plenty of completed applications exist distracting you and lengthening your time to market.

I always like to refer to Guy Kawasaki’s quote “Don’t worry, be crappy”. While I don’t necessarily agree with just throwing functionality out to the www, I believe in quality over quantity, you want to ensure that more time is spent in reviewing the input for new or improved features rather then bugs, bugs, bugs.


Ping.fm and Plurk are two new community driven sites that have leveraged the functionality of other sites, these being Get Satisfaction – People-Powered Customer Service for Everything! and Disq Us – Turn your blog comments into a webwide discussion.

There are advantages and disadvantages to this approach. As an smaller web site with a growing community, exposing what you do to a wider audience when using a third party to manage something can greatly help in exposure and associated marketing at no cost. On the down side, you are losing traffic to another site.

You need to ensure you can always get access to your data, and your community contributions. Ensuring adequate API’s for integration and data extraction are key. From a technology perspective, BitKeeper and LaunchPad come to mind. BitKeeper is a closed source, version control system that MySQL used. This was a killer for community contributions, where individual users simply could not contribute, and if they wanted even access to getting source code via the repository had to pay for an appropriate client. SourceForge and Apache are two examples of huge communities where they leverage the power of the community. LaunchPad is the latest kid on the block, but suffers from the fact that while access to applications hosted there are free, the actually LaunchPad code itself is closed. This has caused some issues.

It’s a fine line, and in the genre of software development, the Internet can create copies of anything just about overnight. More and more I hear about companies working in stealth mode rather then open community input and interaction, but that’s a topic for another discussion.

Time Warner Cable Speed

by

I had my Time Warner cable installed yesterday, a rather painless process. Reported as having Internet speeds of 10MB down and 1/2 MB up, these were confirmed with speedtest.net

Installation – July 5

A day later – Jul 6

Don't use HostMonster

by

Following a 2-4 day outage from my hosting provider of my dedicated server, I decided to move non critical websites to shared hosting. I have one with 1&1 but I created a second account to share load and act as a backup with www.hostmonster.com after a recommendation from a friend. I was able to move stuff, I was able to get some domains there, but it didn’t last long.

What a disaster. NEVER USE HOSTMONSTER!

Probably about a week after my account was created, they decided to move my account, they didn’t notify me it was going to happen, they just did it. They lost all my files, and did not tell me, after making multiple inquires and phone calls.

Here is some history.

Tue, Jun 10, 2008 at 8:59 PM

Dear Hostmonster Customer,

Hostmonster has started migrating your account (ronaldbr).
Below you will find important migration details.  Please refer to your ticket
number 0 for any specific details.


Although Hostmonster will do everything possible to ensure that your
migration goes quickly and smoothly; it is important to understand
that your account will be moving from one physical server location to
another. During migration the IP address attached to your domain name
will be changed from your old server IP to your new server IP. This will
cause a temporary interruption in email, ftp, and the visibility
of your website.


This window of interruption occurs because most Internet Service Providers
(ISP's) take 24-72 hours to clear their Cache.


Although this window could last approximately 24-72 hours it typically
only lasts 48 hours before your site becomes fully functional again. Your
web browser (IE, Firefox, Netscape) has a Cached version of your site
stored on your local system. In some cases it will help if you clear your
browser cache. For more Information about Cache and clearing your browser
Cache please review our article on:

http://helpdesk.hostmonster.com/kb/index.php?x=&mod_id=2&id=352

If after waiting 48 hours and clearing your browsers Cache, your
website has not begun functioning normally please contact our World
Class U.S. based Support Team by phone:

   Hostmonster Support:

       * Main Line: (866) 573-4678

       * Outside U.S: (801) 494-8462

       Support Questions: Press 2


Important Migration Details:

Your username and cPanel password will remain the same.

   * Migration Date: June 10, 2008

   * Migration Start Time: 06:00 PM MST

   * Migration End Time: (estimate) 04:00 AM MST

   * Old Server IP: 74.220.207.97

   * New Server IP:


Check out the CEO's Blog!

Come see the latest news, information, and updates on Hostmonster. While
you're there tell me how you think our company is doing!

Thank you again for choosing HostMonster.Com!

Matt Heaton (CEO)

http://www.mattheaton.com/

I opened a ticket 24 hrs later at Wed Jun 11 2008 09:26PM.

I got a quick response, but was lied too when told no data was lost.

Wed Jun 11 2008 10:02PM by [email protected]
Dear Customer,
Thanks for contacting us.
We apologize for the trouble you've been having, we are working on the issue the migration has some complications none of your data was lost please allow 24 hours tops for the site to be fully functional again.
Thanks
Corbin
Level 1 Support Engineer
hostmonster.com

I asked for a reason why this was done, and why I wasn’t even notified. I was given a lame response with a “isn’t likely” it will happen again, “but might be” both in the same sentence.

Thu Jun 12 2008 10:13PM by [email protected]
We migrated your site to free up hard-disk space on our server, and I apologize that appropriate notice wasn't given. I have notified my supervisor in an effort to recommend improved communications. It isn't likely that your account will be migrated again for our business needs, but it might be needed in the future.

Thank you for your inquiry.

John Pratt
Support Level 1
HostMonster.com
866.573.4678

Probably my second or third call now, is Friday morning, and I’ve had added to my ticket some lame text that it’s being escalated.

Fri Jun 13 2008 08:32AM by [email protected]
I am reopening this ticket. It was supposedly moved from host97 to host262 but cpanel man. is still showing host97 and tracert shows host97. I've tried going to both host262.hostmonster.com:2082 and host97.hostmonster.com:2082 but it's not logging me in. I've tried to change the password and tried both host262.hostmonster.com:2082 and host97.hostmonster.com:2082 with the new password and won't login either. An L3 says the username broke in the migration but it's a different kind of "broken" since it doesn't have the error message in Cpanel Manager. I got L3 approval to move this ticket to the escalations queue.

Nicholas Martin
Support Level 1
HostMonster.com
866.573.4678

So now, it’s Sunday morning, on the phone again, no information, reason or help forthcoming again. A note on ticket Sat 2pm (not visible to me), apparently all files lost on both old server and new server. When were you like going to tell the customer.

This service is woeful, I want my money back.

Of course when I said I wanted to cancel my account and get my money back (I can’t login remember), I was told I would have to call back when the billing department was open.

Well, my complaint will be going to [email protected] – Supervisor of Host Monster Tech Support.

Learning from a Disaster

by

As Farhan has already pointed out to us, Disaster is Inevitable – Must shutdown generators. My primary hosting provider The Planet had a serious meltdown, 9,000 servers unavailable, DNS and administration application .

My server was effectively totally unavailable from 3PM Saturday until 10AM Monday, 43 hours in total.

The problem didn’t stop there. Started and verified servers and domains, but like 8 hours later I find that the DNS is wrong on two important domains (I didn’t discover this because I have them in local /etc/hosts) because I moved them to a different IP like 2 weeks ago.

The Planet denied any problems, ticket logged to get them fixed because admin interface was still down.
Wind forward, Service Updates pages states

June 3 – 3:00pm CDT
All DNS Zone files for ns1, ns2, ns5 and ns6 are completely updated as of the information that was available 5:30PM Central Saturday. All DNS servers have been rebooted and BIND has been restarted..

What a flat out lie. DNS lookup directly against name server confirms it’s wrong. At Wed 12:00am Wednesday, the admin interface finally gives access to view and update zone files. What the. It indicates and IP address which is should be, but clearly not what it is.

I will so be sending a complaint to [email protected] when I finally get this resolved.

And just to make this all worse, my present professional site ronaldbradford.com is of critical importance. It’s my only exposure for preparing to provide information to potential employers, and it’s used for the preparation of consulting information. I’ve been forced earlier today because of events starting today for NY Internet Week to purchase DNS services at www.easydns.com. That didn’t also got to plan, yet another story.

Beyond Blogs

by

I was reading today in a printed magazine Business Week the article Beyond Blogs. It’s unusual these days to actually read on paper what we can find on our online world.

What’s interesting is the printed article did actually contain content I didn’t find online. There was a section called “We Didn’t See ‘em coming”, and it’s finally important site mentioned was iTunes. I found the following comment extremely relevant. “…. But we didn’t guess it would become the leading destination for podcast downloads. Contrary to our expectations, podcasts have evolved into a feature of traditional radio, not a rival to it.”

It’s important that with any business model you know, understand and review consistently your competitors. I find many organizations that don’t do this. You need to know your competitor. But as mentioned with iTunes, the designers of podcasts could have easily considered radio to be a competitor initially. One must always evaluate the changing times regularly.

The following are three more quotes of interest.

“But in the helter-skelter of the blogosphere, we wrote, something important was taking place: In the 10 minutes it took to set up a blogging account, anyone with an Internet connection could become a global publisher. Some could become stars and gain power.”

Like the LAMP stack has done for websites, the cost to entry now to get exposure is very low. The problem is now too much content exists to review, compare and evaluate effectively.

“Turned out it wasn’t quite that simple. The magazine article, archived on our Web site, kept attracting readers and blog links. A few professors worked it into their curricula, sending class after class of students to the story. With all this activity, the piece gained high-octane Google juice.“.

I’d not heard of Google juice before.

In relation to Linked In, FaceBook and MySpace, “While only a small slice of the population wants to blog, a far larger swath of humanity is eager to make friends and contacts, to exchange pictures and music, to share activities and ideas. These social connectors are changing the dynamics of companies around the world. Millions of us are now hanging out on the Internet with customers, befriending rivals,…“.

The next opening keynote – Everything fails, All the Time

by

Our third keynote this morning was by Dr Werner Vogels – CTO Amazon.com

His second question to the audience “How many of you don’t shop at Amazon?” When one or two people raised their hands he commented, “Can I talk to you later.”

He was here to talk about Amazon as the technology company, forget about shopping. Think about the technology that drives that. There were some large and impressive numbers about Amazon. The one stood out was 3.9M shipments for peak day (didn’t mention which day)

Some points from the session.

  • Some Big News — Persistence Storage for Amazon EC2
  • Amazon started as technology consumer, we are now a technology provider.
  • We were enterprise scale, now we are at web scale.
  • Taking the approach, get big fast, the result was every architectural principle was broken.
  • Moving into Software as a Service
  • Develop -> Test -> Operate
  • There is a box between Test and Operate (undifferentiated heavy lifting)
  • We seem to put data centers in the same places as trailer parks.
  • At Amazon we expect our data centers to fail.
  • We expect our software to tolerate these fails.
  • We want to be able to loss a data center and not have this affect our customers.
  • Amazon, brings data centers down multiple times a year, just to show the software survives.
  • The provider model — Scalable, Cost Effective, Reliable, Simple
  • Addressing Uncertainty.

He used the 365 main power failure example. Large Web 2.0 Web sites were down including: Live Journal, Craigslist, Technorati, TypePad, Yelp and USA Today.

Is Amazon a viable option in these times? Worthy of consideration.

Updated You can view more links including video courtesy of Sheeri Kritzer at MySQL Forge Conference Notes

Watching what you say?

by

Marten has opened the 2008 MySQL Conference and Expo this morning in Santa Clara.

What was funny in the early slides was the photo showing the burning of the IPO Prospectus. Marten mentioned now with many Sun lawyers in the audience he has to be more careful what to say.

This morning while coming down for breakfast, a Sun employee entered the lift. I introduced myself, and he indicated he knew me by name. When I asked what department he was in, he said “legal”. Being intrigued as to who he knew me, I’d discovered he has read my emails and blog posts.

I must admit I’ve met a number of new people this week and the first word has been “Oh!”, as in they have heard of my name previously.

These have been unexpected responses and information for me, I’m not normally surprised like this. I’ll not be changing what I say, and how I say it, professionally my writing and publishing will continue to embody “Opinions, Expertise, Passion”.

Share/Add This Buttons on sites

by

I’ve noticed a change of buttons lately on sites where you can bookmark/share the relevant information. So I’ve done a cross sample of A Computer Site – www.dell.com, The MySQL Acquirer – www.sun.com, A News site – www.cnn.com and a technology information site – www.techcrunch.com

What got me to go back and research is I’d never seen a FaceBook icon before, or perhaps I’d never paid enough attention.

At the end most sites now wash out to a site called Add This.










Microsoftism's on my MacBook

by

Seems Mac OS/X has not escaped the unnecessary annoyances of Windows. Below are details to unstall a product on my MacBook, following it crashing my machine a few times with the “blue screen on death” below.

Eye TV Lite is the software that comes with the Pinnacle TV line of products for the Apple Mac.

 

To completely uninstall the EyeTV Lite software and it associated files, please follow the steps below.

 

1.      Drag the EyeTV Lite application from the Application folder to the trash.

a.       Open Finder

b.      Select Applications

c.       Select EyeTV Lite and drab it to the trash

 

2.      Using Finder, move the following files to the trash.

 

Below is a list of the files that need to be removed and their function.  Below that is the location of the files. 

 

com.elgato.eyetv.plist – stores the preferences and there are 2 copies on the hard drive

EyeTV Helper – assits with USB/FireWire issues

com.elgato.eyetv.devices.plist – preference file related to the available devices

com.elgato.eyetv.world.plist – preference file related to the channel lists

Wakein – assists EyeTV Lite with waking/booting to record

EyeTVEPG.db – EPG data is stored in this file

EyeTV Archive – contains all of your recordings and scheduled programs

EyeTVClassicDontSeize.kext – All of the kernel extension files .kext are to prevent the Mac OS X from attempting to talk to TV devices that can appear as generic USB audio class or HID class devices before the actual firmware has been uploaded to them.

EyeTVAfaTechHidBlock.kext – same as above

            EyeTVEmpiaAudioBlock.kext – same as above

 

Drag these files from the location listed to the trash using Finder

 

/Users/(username)/Library/Preferences/com.elgato.eyetv.plist

 

/Library/Preferences/com.elgato.eyetv.plist

/Library/Preferences/com.elgato.eyetv.devices.plist

/Library/Preferences/com.elgato.eyetv.world.plist

 

/Library/Application Support/EyeTV/EyeTV Helper

/Library/Application Support/EyeTV/Wakein

/Library/Application Support/EyeTV/EyeTVEPG.db

 

/Users/(username)/Library/EyeTV Archive/

 

/System/Library/Extensions/EyeTVClassicDontSeize.kext

/System/Library/Extensions/EyeTVAfaTechHidBlock.kext

/System/Library/Extensions/EyeTVEmpiaAudioBlock.kext

 

NOTE 1:  If you erase the EyeTV preference files, then any activation key you may have will need to be re-entered.

 

NOTE 2:  The files with the extention .kext can only be erased with root/superuser access. Also, to erase EyeTV Helper, you first have to quit it by using the Process Viewer or Activity Monitor.

Reference

The fobar of a Web 2.0 website

by

Web 2.0 is all about community driven content. Recently eBay purchased Stumble Upon for $75 million. There is a problem here. When I first heard of the site, I looked at. I remember going back the next few days, and I was sure it hadn’t changed. Then I started taking screen shots. Having forgotten about it now for over a month under a discussion today, I took another screenshot. In over a month, from June 3rd to July 15th there has been no content change to the website. There has been a change to the counter of number of stumblers, and a change of image for a Recent Stumblers, but the “content”, the recent popular web site on the first page of the website remains unchanged. Check out my screenshots.

-rw-r--r-- 1 rbradford rbradford 568715 2007-07-15 19:13 stumbleapon.15072007.png
-rw-r--r-- 1 rbradford rbradford 570622 2007-06-03 21:32 stumbleupon.03062007.png
-rw-r--r-- 1 rbradford rbradford 580385 2007-06-04 18:25 stumbleupon.04062007.png
-rw-r--r-- 1 rbradford rbradford 586082 2007-06-07 00:22 stumbleupon.07062007.png
-rw-r--r-- 1 rbradford rbradford 570792 2007-06-15 02:03 stumbleupon.15062007.png

Truemors, Tumors, Dribble

by

I was sent this email.


http://blog.guykawasaki.com/2007/06/by_the_numbers_.html

By the Numbers: How I built a Web 2.0, User-Generated Content, Citizen – Journalism, Long-Tail, Social Media Site for $12,107.09

I have had my flatmate talk about it a lot in the past day. So I checked it out. Here is my view.

My Review

I’ve heard the name “Guy Kawasaki” but never read his stuff, or followed him. My first impression was the 2007 MySQL Conference and I was suitably impressed. See MySQL Conference – The next keynote with Guy Kawasaki

So some of my comments following this site — Truemors.

$1,115.05. I spent $1,115.05 registering domains. I could have used GoDaddy and done it a lot cheaper, but I was too stupid and lazy.

55. I registered 55 domains (for example, truemors.net, .de, .biz, truemours, etc, etc). I had no idea that one had to buy so many domains to truly “surround” the one you use. Yes, I could have registered fewer and spent less, but who cares about saving a few hundred bucks compared to the cost of legal action to get a domain away from a squatter if Truemors is successful?

Me: Your an idiot, and I don’t use the term lightly. I had great respect after one presentation, I even referred to his talk last night at a meeting for people to improve their presentations, but you paid 3 times the price for one domain, ok, but for 55, and then he hasn’t covered the name to well. A few quick typos and people have already registered names and stuck up google ads.

4. I learned four lessons launching Truemors:
There’s really no such thing as bad PR.
$12,000 goes a very long way these days.
You can work with a team that is thousands of miles away.
Life is good for entrepreneurs these days.

Me: There is such a thing as bad PR if your not “Guy Kawasaki” a name that draws a crowd.
$12,000 does go a long way if you spend it wisely. $399 for a logo, I could have done in 10 mins in Photoshop, and I’m an amateur. I’ve got better free logos on my sites. You have to get to being a successful entrepreneur before you can say “life is good for entrepreneurs these days”

Conclusion

I could make a comment on a number of his points.
My conclusion is he certainly polarized people. The responses are either completely for, or completely against.

If you want to read on, here are a few comments I’ve taken from his own blog comments which I found interesting. For those lazy and just like scanning.
Oh, and if you go to his site, the top reference on his own site is people bagging his site. Yeah that’s really good PR – Screen print at the end.

=========================================================================
Poor Guy! I have a number for you: number of websites needed to destroy your reputation: 1.

Wait, a minute you’re an “expert” on startups, and you spent more than a $1000 on domain name registration? Hahaha! LMAO

To me, Truemors would have been better if you:

1) Spent some money to have some juicy content prepared for launch to be submitted by random people.
2) Focused the product to a specific topic. Digg started out with technology only. Truemors could do celebrity gossip?

OK, here’s a “truemor”: I just scanned the comments below and Ray’s Jun 4, 2007 12:33:34 AM posting was so LOL funny *I* damn near messed myself. :-)

$4,824.14 for legal fees… What did the lawyers do for you? Did they come up with your terms of use? Do you have a privacy statement? Do tell!

Interesting numbers – but totally irrelevant! The whole point of business is to make money, not how carefully you spend what money you have! It would seem to me that Web 2.0 is all about hype – getting cheap eyeballs and selling them! Could you repost in a month or so and let us know when you’ve started making money?

You’ve proven you can start a web2.0 site for under 15k? WTF. Are you serious or are you just in your 30’s and senile? Most “successful” web2.0 sites get off the ground with less than $1,000.00. Truemors will be a graveyard in a month… might as well throw in the towel now.

You paid $399 for that logo!?? You could have told me…I would have created something better than that for free… Honestly, most of the things look a huge waste of money..
Guy, I guess this might offend you, but lately it seems you’ve been investing in some bad ideas. The idea for your new site is “okay”, however the layout is so terrible that the first time I saw the site, it was so annoying I skipped the article that brought me there and closed my browser tab.

My ‘hourly’ MySQL monitor script Version 0.03

by

I realized when I released my very crappy version of My ‘hourly’ MySQL monitor script I really should have included my standard logging.

So I did that the night I wrote my original blog, but never published it. I’ve had need to use it again today, so a few more usability tweaks for parameterization and we are good to go.

Now Version 0.03 includes three files:

  • hourly.sh
  • common.sh
  • mysql.conf

Simple use is:

$ cd /directory
$ vi mysql.conf
# correctly specify MYSQL_AUTHENTICATION
$ chmod +x ./hourly.sh
$ nohup hourly.sh &

This gives you the following files

-rw-r--r-- 1 rbradford rbradford  2643 2007-05-29 15:47 mysql.innodbstatus.070529.154757.log
-rw-r--r-- 1 rbradford rbradford   414 2007-05-29 15:47 mysql.processlist.070529.154757.log
-rw-r--r-- 1 rbradford rbradford 12597 2007-05-29 15:47 mysql.status.070529.154757.log
-rw-r--r-- 1 rbradford rbradford 22229 2007-05-29 15:47 mysql.variables.070529.154757.log
-rw-r--r-- 1 rbradford rbradford 13146 2007-05-29 15:47 os.ps.070529.154757.log
-rw-r--r-- 1 rbradford rbradford   390 2007-05-29 15:48 os.vmstat.5.070529.154757.log

By default, written in /tmp, you can override by setting LOG_DIR.

It gives you a pile of output you can easily grep, I’m working on some very simple graphing. One thing I have done is pass the status into Mark Leith’s Aggregating SHOW STATUS Output as well as passed on some feedback that I hope will get integrated into later solutions.

For now, it’s a tool I can implement in a few seconds, run while somebody is showing or demonstrating a system, and I’ve got some meaningful information to look at. Combined with my more in-depth ‘minute’ script, a general-log and taking notes of individual steps in a system walk though, I have all the information I need to analyze a working system very quickly from a purely database level. Still there is lots to do manually, but I’ve got a consistent view of information to review.

New Category – Clever Design

by

I am sometimes really impressed by Clever Design in print media or TV media. Over the years I’ve even collected piles of ideas, who knows where they all are now. Time to consolidate into one area, a new category on my Blog aptly named “Clever Design”. A place where I can put ideas, images, marketing, TV ads etc that have had a positive effect on me anyway. Stay tuned!