Mysql – What’s the approach to figure out reason for thesql crash

crashinnodbMySQL

The database I'm using crashes a lot recently. And I can't find a decent way to dig into the issue. I looked into mysql error log, but don't know how to start. I know there was a "load data infile" query running when it crashes. Sometimes the table is corrupted so I have to drop it and rebuild, sometimes the table is just fine (both cases are when the "load data infile" is running).

This happened many times and I've recovered the database several times. But it keeps happening and I want to find out the reason and resolve the cause. I've follow the recovery link, and I've tried to search around the failure message. But couldn't figure it out..

So what will you do to find out the crash cause? Is there a guideline? Really appreciate it.

2014-12-08 11:01:52 30924 [Note] InnoDB: Resuming purge

2014-12-08 11:59:10 7f945e0e3700 InnoDB: Assertion failure in thread 140275209877248 in file buf0buf.cc line 3687

InnoDB: Failing assertion: buf_fix_count > 0

InnoDB: We intentionally generate a memory trap.

InnoDB: Submit a detailed bug report to (a link was here, I don't have enough reputation to post links yet).

InnoDB: If you get repeated assertion failures or crashes, even

InnoDB: immediately after the mysqld startup, there may be

InnoDB: corruption in the InnoDB tablespace. Please refer to (a link was here, I don't have enough reputation to post links yet)

InnoDB: about forcing recovery.

19:59:10 UTC – mysqld got signal 6 ;

This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Please help us make Percona Server better by reporting any bugs at percona bugs webiste (a link was here, I don't have enough reputation to post links yet)

key_buffer_size=33554432

read_buffer_size=131072

max_used_connections=6

max_threads=1002

thread_count=3

connection_count=3

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 432019 K bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0

Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong…

stack_bottom = 0 thread_stack 0x40000

/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x8c821e]

/usr/sbin/mysqld(handle_fatal_signal+0x4a4)[0x671f44]

/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7f98c3646cb0]

/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7f98c2a9c425]

/lib/x86_64-linux-gnu/libc.so.6(abort+0x17b)[0x7f98c2a9fb8b]

/usr/sbin/mysqld[0xa5e1c8]

/usr/sbin/mysqld[0xa754ce]

/usr/sbin/mysqld[0xa766b3]

/usr/sbin/mysqld[0xa5a727]

/usr/sbin/mysqld[0xa3de9c]

/usr/sbin/mysqld[0x9e384d]

/usr/sbin/mysqld[0x9e0b7d]

/usr/sbin/mysqld[0x9e1ecc]

/usr/sbin/mysqld[0x9b299c]

/usr/sbin/mysqld[0xa09f26]

/usr/sbin/mysqld[0x9fc62c]

/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f98c363ee9a]

/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f98c2b59ccd]

You may download the Percona Server operations manual by visiting (a link was here, I don't have enough reputation to post links yet). You may find information in the manual which will help you identify the cause of the crash.

141208 11:59:14 mysqld_safe Number of processes running now: 0

141208 11:59:14 mysqld_safe mysqld restarted

Best Answer

This appears to be a regression bug that exists in Percona Server 5.6.14 and possibly some previous versions of 5.6.

https://bugs.launchpad.net/percona-server/+bug/1247021

It appears it was fixed in Percona Server 5.6.16-64.0:

Due to a regression in the buffer pool mutex split, a combination of InnoDB compression, write workload, and multiple active purge threads could lead to a server crash. Bug fixed #1247021.

http://www.percona.com/doc/percona-server/5.6/release-notes/Percona-Server-5.6.16-64.0.html

Related Question