Ronald Bradford
MySQL Expert

MySQL Expert Ronald Bradford shares valuable input in MySQL Performance Tuning, MySQL Scalability and general MySQL Help from his two decades of working with MySQL, Oracle, Ingres and development technologies.

«
»

Unexpected mysqld crashing in 5.5

An update of MySQL from 5.0 to 5.5 on CentOS 5.5 64bit has not resulted in a good experience. The mysqld process would then crash every few minutes with the following message.

101120 8:29:27 InnoDB: Operating system error number 22 in a file operation.
InnoDB: Error number 22 means ‘Invalid argument’.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/operating-system-error-codes.html
InnoDB: File name /tmpfs/#sql6cf3_5c_0.ibd
InnoDB: File operation call: ‘aio write’.
InnoDB: Cannot continue operation.

The work around was to change the tmpdir=/tmpfs (which was a 16G tmpfs filesystem) to a physical disk.

The referenced URL didn’t provide any more information of help. Unlike Bug #26662 O_DIRECT is not specified as the flush method.

Tags: , , , ,

This entry was posted on Monday, November 22nd, 2010 at 9:41 am and is filed under Databases, MySQL, Professional. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

8 Responses to “Unexpected mysqld crashing in 5.5”

  1. Morgan Tocker says:
    November 22, 2010 at 10:36 am

    aio = http://en.wikipedia.org/wiki/Asynchronous_I/O – a new feature of 5.5. Try setting innodb_use_native_aio=0.

    If it works, file a bug saying tmpfs does not support aio, and InnoDB should revert back to the simulated Asynchronous I/O.

  2. Mark R says:
    November 22, 2010 at 11:33 am

    I have reproduced this; steps to reproduce

    1. Set tmpdir to a tmpfs on Linux
    2. Create an innodb temporary table

    Reproducable crash. I don’t want to steal your thunder, you report it.

  3. Mark R says:
    November 22, 2010 at 11:37 am

    It seems a shame they did not add this test case to their test suite after Bug #26662 you mentioned above; the same circumstances trigger it.

  4. Mike says:
    November 22, 2010 at 1:27 pm

    AIO on tmpfs requires Linux > 2.6.22. Update to a modern kernel or use ramdisk.

  5. ronald says:
    November 22, 2010 at 4:19 pm

    Thanks Mike@ I can’t contact you.

    Kernel is 2.6.18-194.26.1.el5 so I’ll add that to the list to try with the client. Second new thing I learned today.

  6. Ovais Tariq says:
    November 23, 2010 at 8:36 am

    That has something to do with the native AIO support in the newer InnoDB shipped with MySQL 5.5, though I have MySQL 5.5 installed on Ubuntu 10.04.1 LTS (Lucid Lynx) and its working perfectly fine,.

  7. Mark R says:
    November 23, 2010 at 12:29 pm

    This is the kernel shipped with Redhat Enterprise Linux 5, a recent, widely used and well-supported OS. MySQL has the bug and it should be fixed.

  8. Raghavendra Prabhu says:
    December 2, 2011 at 6:49 pm

    Simple fix for this:
    http://bazaar.launchpad.net/~raghavendra-prabhu/+junk/mysql-server-fallocate/revision/3553 . I have tested it only to the point of building it, however, anyone can test it on older kernels or incompatible AIO sinks and let me know.

Leave a Reply