Unexpected mysqld crashing in 5.5

An update of MySQL from 5.0 to 5.5 on CentOS 5.5 64bit has not resulted in a good experience. The mysqld process would then crash every few minutes with the following message.

101120 8:29:27 InnoDB: Operating system error number 22 in a file operation.
InnoDB: Error number 22 means ‘Invalid argument’.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/operating-system-error-codes.html
InnoDB: File name /tmpfs/#sql6cf3_5c_0.ibd
InnoDB: File operation call: ‘aio write’.
InnoDB: Cannot continue operation.

The work around was to change the tmpdir=/tmpfs (which was a 16G tmpfs filesystem) to a physical disk.

The referenced URL didn’t provide any more information of help. Unlike Bug #26662 O_DIRECT is not specified as the flush method.

Tags: , , , ,

8 Responses to “Unexpected mysqld crashing in 5.5”

  1. Morgan Tocker says:

    aio = http://en.wikipedia.org/wiki/Asynchronous_I/O – a new feature of 5.5. Try setting innodb_use_native_aio=0.

    If it works, file a bug saying tmpfs does not support aio, and InnoDB should revert back to the simulated Asynchronous I/O.

  2. Mark R says:

    I have reproduced this; steps to reproduce

    1. Set tmpdir to a tmpfs on Linux
    2. Create an innodb temporary table

    Reproducable crash. I don’t want to steal your thunder, you report it.

  3. Mark R says:

    It seems a shame they did not add this test case to their test suite after Bug #26662 you mentioned above; the same circumstances trigger it.

  4. Mike says:

    AIO on tmpfs requires Linux > 2.6.22. Update to a modern kernel or use ramdisk.

  5. ronald says:

    Thanks Mike@ I can’t contact you.

    Kernel is 2.6.18-194.26.1.el5 so I’ll add that to the list to try with the client. Second new thing I learned today.

  6. Ovais Tariq says:

    That has something to do with the native AIO support in the newer InnoDB shipped with MySQL 5.5, though I have MySQL 5.5 installed on Ubuntu 10.04.1 LTS (Lucid Lynx) and its working perfectly fine,.

  7. Mark R says:

    This is the kernel shipped with Redhat Enterprise Linux 5, a recent, widely used and well-supported OS. MySQL has the bug and it should be fixed.

  8. Simple fix for this:
    http://bazaar.launchpad.net/~raghavendra-prabhu/+junk/mysql-server-fallocate/revision/3553 . I have tested it only to the point of building it, however, anyone can test it on older kernels or incompatible AIO sinks and let me know.