Crash confirmed with latest bzr trunk (290.2.6) with runall.pl and details provided above. Creating Directories Installing Master Database ======================================================= TEST RESULT TIME (ms) ------------------------------------------------------- Servers started, exiting # 2012-09-06T01:38:09 Starting: gentest.pl --gendata=/home/raghu/assert.zz --engine=InnoDB --threads=4 --queries=10M --duration=600 --dsn=dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test --grammar=/home/raghu/assert.yy # 2012-09-06T01:38:09 ------------------------------- # 2012-09-06T01:38:09 Configuration # 2012-09-06T01:38:09 dsn => ['dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test'] # 2012-09-06T01:38:09 duration => 600 # 2012-09-06T01:38:09 engine => InnoDB # 2012-09-06T01:38:09 gendata => /home/raghu/assert.zz # 2012-09-06T01:38:09 generator => FromGrammar # 2012-09-06T01:38:09 grammar => /home/raghu/assert.yy # 2012-09-06T01:38:09 queries => 10M # 2012-09-06T01:38:09 reporters => [''] # 2012-09-06T01:38:09 seed => 1 # 2012-09-06T01:38:09 threads => 4 # 2012-09-06T01:38:09 transformers => [''] # 2012-09-06T01:38:09 validators => [''] # 2012-09-06T01:38:09 # Creating MySQL table: test.table10_innodb_key_pk_parts_2_int_autoinc; engine: innodb; rows: 10 . # 2012-09-06T01:38:09 # Creating MySQL table: test.table100_innodb_key_pk_parts_2_int_autoinc; engine: innodb; rows: 100 . # 2012-09-06T01:38:09 Loading Generator GenTest::Generator::FromGrammar. # 2012-09-06T01:38:10 Caching schema metadata for dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test # 2012-09-06T01:38:10 Reporters: ErrorLog, Backtrace # 2012-09-06T01:38:10 Validators: ErrorMessageCorruption # 2012-09-06T01:38:10 Starting 4 processes, 10M queries each, duration 600 seconds. # 2012-09-06T01:38:10 GenTest::ErrorFilter(2627) started # 2012-09-06T01:38:10 Loading Validator GenTest::Validator::ErrorMessageCorruption. # 2012-09-06T01:38:10 Loading Validator GenTest::Validator::ErrorMessageCorruption. # 2012-09-06T01:38:11 Loading Validator GenTest::Validator::ErrorMessageCorruption. # 2012-09-06T01:38:11 Loading Validator GenTest::Validator::ErrorMessageCorruption. # 2012-09-06T01:38:11 Started periodic reporting process... # 2012-09-06T01:42:44 Query: SHOW ENGINE INNODB STATUS failed: 2013 Lost connection to MySQL server during query # 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test # 2012-09-06T01:42:44 Query: UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 4 failed: 2013 Lost connection to MySQL server during query # 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test # 2012-09-06T01:42:44 Query: UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 1 failed: 2013 Lost connection to MySQL server during query # 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test # 2012-09-06T01:42:44 Query: UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 7 failed: 2013 Lost connection to MySQL server during query # 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test # 2012-09-06T01:42:44 Killing child process with pid 2631... # 2012-09-06T01:42:44 Killing child process with pid 2635... # 2012-09-06T01:42:44 Killing child process with pid 2633... # 2012-09-06T01:42:45 Killing periodic reporting process with pid 2628... # 2012-09-06T01:42:45 Kill GenTest::ErrorFilter(2627) # 2012-09-06T01:42:45 Server crash reported, initiating post-crash analysis... # 2012-09-06T01:42:45 The last 100 lines from /home/raghu/mysql/master-data//../log/master.err : InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1. InnoDB: DEBUG: update_statistics for test/table10_innodb_key_pk_parts_2_int_autoinc#P#p0. InnoDB: DEBUG: update_statistics for test/table10_innodb_key_pk_parts_2_int_autoinc#P#p1. 120905 23:12:37 InnoDB: Assertion failure in thread 140543818106624 in file rem0rec.ic line 973 InnoDB: Failing assertion: (ulint) index == offsets[3] InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 20:12:37 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Please help us make Percona Server better by reporting any bugs at http://bugs.percona.com/ key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=5 max_threads=151 thread_count=5 connection_count=5 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 60862 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2ced230 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7fd2e85a7e68 thread_stack 0x40000 /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(my_print_stacktrace+0x2e)[0x7d7195] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(handle_fatal_signal+0x2f1)[0x6ad0a1] /lib/libpthread.so.0(+0xf170)[0x7fd2ec09f170] /lib/libc.so.6(gsignal+0x35)[0x7fd2ea655fa5] /lib/libc.so.6(abort+0x148)[0x7fd2ea657428] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x986ff5] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x98a017] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x990a5a] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x88f112] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x839db7] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z14ha_show_statusP3THDP10handlerton12ha_stat_type+0x3fe)[0x6b5876] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z21mysql_execute_commandP3THD+0xe9b)[0x595bfa] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x223)[0x59c7ea] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x7dd)[0x59d28a] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z10do_commandP3THD+0x282)[0x59e75f] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z24do_handle_one_connectionP3THD+0x10c)[0x648061] /media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(handle_one_connection+0x52)[0x648157] /lib/libpthread.so.0(+0x7e0f)[0x7fd2ec097e0f] /lib/libc.so.6(clone+0x6d)[0x7fd2ea70504d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7fd2d000e780): SHOW ENGINE INNODB STATUS Connection ID (thread ID): 7 Status: NOT_KILLED You may download the Percona Server operations manual by visiting http://www.percona.com/software/percona-server/. You may find information in the manual which will help you identify the cause of the crash. Writing a core file # 2012-09-06T01:42:45 datadir is /home/raghu/mysql/master-data/ # 2012-09-06T01:42:45 binary is /media/Vone/percona-server/raghu/Percona-Server/mysql-test/../sql/mysqld # 2012-09-06T01:42:45 bindir is /media/Vone/percona-server/raghu/Percona-Server/mysql-test/../sql # 2012-09-06T01:42:45 core is /home/raghu/mysql/master-data/core warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fffe276f000 15 /media/Vone/percona-server/origin/Percona-Server/sql/main.cc: No such file or directory. # 2012-09-06T01:42:57 [New LWP 2630] # 2012-09-06T01:42:57 [New LWP 2636] # 2012-09-06T01:42:57 [New LWP 2598] # 2012-09-06T01:42:57 [New LWP 2600] # 2012-09-06T01:42:57 [New LWP 2601] # 2012-09-06T01:42:57 [New LWP 2603] # 2012-09-06T01:42:57 [New LWP 2604] # 2012-09-06T01:42:57 [New LWP 2608] # 2012-09-06T01:42:57 [New LWP 2597] # 2012-09-06T01:42:57 [New LWP 2611] # 2012-09-06T01:42:57 [New LWP 2610] # 2012-09-06T01:42:57 [New LWP 2593] # 2012-09-06T01:42:57 [New LWP 2606] # 2012-09-06T01:42:57 [New LWP 2615] # 2012-09-06T01:42:57 [New LWP 2602] # 2012-09-06T01:42:57 [New LWP 2634] # 2012-09-06T01:42:57 [New LWP 2609] # 2012-09-06T01:42:57 [New LWP 2632] # 2012-09-06T01:42:57 [New LWP 2596] # 2012-09-06T01:42:57 [New LWP 2607] # 2012-09-06T01:42:57 [New LWP 2595] # 2012-09-06T01:42:57 [New LWP 2599] # 2012-09-06T01:42:57 [New LWP 2612] # 2012-09-06T01:42:57 [Thread debugging using libthread_db enabled] # 2012-09-06T01:42:57 Using host libthread_db library "/lib/libthread_db.so.1". # 2012-09-06T01:42:57 Core was generated by `/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld --no-defaults --base'. # 2012-09-06T01:42:57 Program terminated with signal 6, Aborted. # 2012-09-06T01:42:57 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0 # 2012-09-06T01:42:57 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0 # 2012-09-06T01:42:57 #1 0x00000000007d6c58 in my_write_core (sig=6) at /media/Vone/percona-server/origin/Percona-Server/mysys/stacktrace.c:433 # 2012-09-06T01:42:57 #2 0x00000000006ad21c in handle_fatal_signal (sig=6) at /media/Vone/percona-server/origin/Percona-Server/sql/signal_handler.cc:249 # 2012-09-06T01:42:57 #3 # 2012-09-06T01:42:57 #4 0x00007fd2ea655fa5 in raise () from /lib/libc.so.6 # 2012-09-06T01:42:57 #5 0x00007fd2ea657428 in abort () from /lib/libc.so.6 # 2012-09-06T01:42:57 #6 0x0000000000986ff5 in rec_offs_validate (rec=rec@entry=0x7fd2e3d904ec "\200", index=index@entry=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/rem0rec.ic:973 # 2012-09-06T01:42:57 #7 0x000000000098a017 in lock_rec_queue_validate (block=block@entry=0x7fd2dbfe9740, rec=rec@entry=0x7fd2e3d904ec "\200", index=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4756 # 2012-09-06T01:42:57 #8 0x0000000000990a5a in lock_rec_validate_page (block=0x7fd2dbfe9740) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4965 # 2012-09-06T01:42:57 #9 lock_rec_block_validate (page_no=, space=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5056 # 2012-09-06T01:42:57 #10 lock_validate () at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5109 # 2012-09-06T01:42:57 #11 lock_print_info_all_transactions (file=file@entry=0x294d5a0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4580 # 2012-09-06T01:42:57 #12 0x000000000088f112 in srv_printf_innodb_monitor (file=0x294d5a0, nowait=0, trx_start=0x7fd2e85a6780, trx_end=0x7fd2e85a6788) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2243 # 2012-09-06T01:42:57 #13 0x0000000000839db7 in innodb_show_status (stat_print=0x6ad7aa , thd=0x2ced230, hton=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10374 # 2012-09-06T01:42:57 #14 innobase_show_status (hton=, thd=0x2ced230, stat_print=0x6ad7aa , stat_type=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10614 # 2012-09-06T01:42:57 #15 0x00000000006b5876 in ha_show_status (thd=0x2ced230, db_type=0x254adb0, stat=HA_ENGINE_STATUS) at /media/Vone/percona-server/origin/Percona-Server/sql/handler.cc:5066 # 2012-09-06T01:42:57 #16 0x0000000000595bfa in mysql_execute_command (thd=thd@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2480 # 2012-09-06T01:42:57 #17 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2ced230, rawbuf=, length=25, parser_state=parser_state@entry=0x7fd2e85a7760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811 # 2012-09-06T01:42:57 #18 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2ced230, packet=packet@entry=0x2d78151 "", packet_length=packet_length@entry=25) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061 # 2012-09-06T01:42:57 #19 0x000000000059e75f in do_command (thd=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788 # 2012-09-06T01:42:57 #20 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484 # 2012-09-06T01:42:57 #21 0x0000000000648157 in handle_one_connection (arg=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391 # 2012-09-06T01:42:57 #22 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:57 #23 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:57 #5 0x00007fd2ea657428 in abort () from /lib/libc.so.6 warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fffe276f000 # 2012-09-06T01:42:59 [New LWP 2630] # 2012-09-06T01:42:59 [New LWP 2636] # 2012-09-06T01:42:59 [New LWP 2598] # 2012-09-06T01:42:59 [New LWP 2600] # 2012-09-06T01:42:59 [New LWP 2601] # 2012-09-06T01:42:59 [New LWP 2603] # 2012-09-06T01:42:59 [New LWP 2604] # 2012-09-06T01:42:59 [New LWP 2608] # 2012-09-06T01:42:59 [New LWP 2597] # 2012-09-06T01:42:59 [New LWP 2611] # 2012-09-06T01:42:59 [New LWP 2610] # 2012-09-06T01:42:59 [New LWP 2593] # 2012-09-06T01:42:59 [New LWP 2606] # 2012-09-06T01:42:59 [New LWP 2615] # 2012-09-06T01:42:59 [New LWP 2602] # 2012-09-06T01:42:59 [New LWP 2634] # 2012-09-06T01:42:59 [New LWP 2609] # 2012-09-06T01:42:59 [New LWP 2632] # 2012-09-06T01:42:59 [New LWP 2596] # 2012-09-06T01:42:59 [New LWP 2607] # 2012-09-06T01:42:59 [New LWP 2595] # 2012-09-06T01:42:59 [New LWP 2599] # 2012-09-06T01:42:59 [New LWP 2612] # 2012-09-06T01:42:59 [Thread debugging using libthread_db enabled] # 2012-09-06T01:42:59 Using host libthread_db library "/lib/libthread_db.so.1". # 2012-09-06T01:42:59 Core was generated by `/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld --no-defaults --base'. # 2012-09-06T01:42:59 Program terminated with signal 6, Aborted. # 2012-09-06T01:42:59 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 23 (Thread 0x7fd2e85e9700 (LWP 2612)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09eea9 in do_sigwait () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00007fd2ec09ef3b in sigwait () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #2 0x0000000000516ab1 in signal_hand (arg=) at /media/Vone/percona-server/origin/Percona-Server/sql/mysqld.cc:2732 # 2012-09-06T01:42:59 #3 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #4 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 22 (Thread 0x7fd2dab5d700 (LWP 2599)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=2, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=4, message1=message1@entry=0x7fd2dab5ce68, message2=message2@entry=0x7fd2dab5ce60, type=type@entry=0x7fd2dab5ce58, space_id=space_id@entry=0x7fd2dab5ce50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=4) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 21 (Thread 0x7fd2ea406700 (LWP 2595)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x294ec10) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=0, message1=message1@entry=0x7fd2ea405e68, message2=message2@entry=0x7fd2ea405e60, type=type@entry=0x7fd2ea405e58, space_id=space_id@entry=0x7fd2ea405e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 20 (Thread 0x7fd2d6caa700 (LWP 2607)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09bcc1 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000009b04cd in os_cond_wait_timed (abstime=0x7fd2d6ca9da0, mutex=0x27fa3e0, cond=0x27fa418) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:171 # 2012-09-06T01:42:59 #2 os_event_wait_time_low (event=0x27fa3e0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:716 # 2012-09-06T01:42:59 #3 0x0000000000890c18 in srv_error_monitor_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2887 # 2012-09-06T01:42:59 #4 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #5 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 19 (Thread 0x7fd2e8dea700 (LWP 2596)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x295ce90) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=1, message1=message1@entry=0x7fd2e8de9e68, message2=message2@entry=0x7fd2e8de9e60, type=type@entry=0x7fd2e8de9e58, space_id=space_id@entry=0x7fd2e8de9e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 18 (Thread 0x7fd2e8526700 (LWP 2632)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207 # 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609 # 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=2) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458 # 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453 # 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419 # 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=0, pass@entry=140543342471336, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779 # 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942 # 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59 # 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=page@entry=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc016cb8, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252 # 2012-09-06T01:42:59 #10 0x00000000008dd45d in btr_cur_search_to_nth_level (index=index@entry=0x7fd2d001b728, level=level@entry=0, tuple=tuple@entry=0x7fd2cc02a4e8, mode=4, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc016cb8, has_search_latch=has_search_latch@entry=0, file=file@entry=0xb1b1c0 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c", line=line@entry=3097, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:756 # 2012-09-06T01:42:59 #11 0x00000000008e9279 in btr_pcur_open_with_no_init_func (mtr=0x7fd2e8523dc0, line=3097, file=0xb1b1c0 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c", has_search_latch=0, cursor=0x7fd2cc016cb8, latch_mode=1, mode=, tuple=0x7fd2cc02a4e8, index=0x7fd2d001b728) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:482 # 2012-09-06T01:42:59 #12 btr_pcur_restore_position_func (latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc016cb8, file=file@entry=0xb1b1c0 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c", line=line@entry=3097, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0pcur.c:351 # 2012-09-06T01:42:59 #13 0x0000000000870b0e in sel_restore_position_for_mysql (same_user_rec=same_user_rec@entry=0x7fd2e85242a0, latch_mode=latch_mode@entry=1, pcur=pcur@entry=0x7fd2cc016cb8, moves_up=moves_up@entry=1, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:3097 # 2012-09-06T01:42:59 #14 0x0000000000878963 in row_search_for_mysql (buf=0x7fd2cc012008 "\377", mode=1, prebuilt=0x7fd2cc016c48, match_mode=0, direction=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:4759 # 2012-09-06T01:42:59 #15 0x00000000008349ae in ha_innobase::index_read (this=0x7fd2cc0123e0, buf=0x7fd2cc012008 "\377", key_ptr=, key_len=0, find_flag=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:6665 # 2012-09-06T01:42:59 #16 0x0000000000832071 in ha_innobase::index_first (this=0x7fd2cc0123e0, buf=0x7fd2cc012008 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7015 # 2012-09-06T01:42:59 #17 0x000000000083f703 in ha_innobase::rnd_next (this=0x7fd2cc0123e0, buf=0x7fd2cc012008 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7112 # 2012-09-06T01:42:59 #18 0x0000000000a18506 in ha_partition::rnd_next (this=0x7fd2cc011aa0, buf=0x7fd2cc012008 "\377") at /media/Vone/percona-server/origin/Percona-Server/sql/ha_partition.cc:4032 # 2012-09-06T01:42:59 #19 0x0000000000777233 in rr_sequential (info=0x7fd2e8524590) at /media/Vone/percona-server/origin/Percona-Server/sql/records.cc:455 # 2012-09-06T01:42:59 #20 0x0000000000613f85 in mysql_update (thd=0x2d83e40, table_list=0x7fd2cc004e38, fields=..., values=..., conds=0x0, order_num=, order=0x0, limit=18446744073709551615, handle_duplicates=DUP_ERROR, ignore=false, found_return=0x7fd2e85248f0, updated_return=0x7fd2e85248f8) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_update.cc:653 # 2012-09-06T01:42:59 #21 0x000000000059785c in mysql_execute_command (thd=thd@entry=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2940 # 2012-09-06T01:42:59 #22 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2d83e40, rawbuf=, length=73, parser_state=parser_state@entry=0x7fd2e8525760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811 # 2012-09-06T01:42:59 #23 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2d83e40, packet=packet@entry=0x2d87581 "UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 4", packet_length=packet_length@entry=73) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061 # 2012-09-06T01:42:59 #24 0x000000000059e75f in do_command (thd=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788 # 2012-09-06T01:42:59 #25 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484 # 2012-09-06T01:42:59 #26 0x0000000000648157 in handle_one_connection (arg=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391 # 2012-09-06T01:42:59 #27 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #28 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 17 (Thread 0x7fd2d5ca8700 (LWP 2609)): # 2012-09-06T01:42:59 #0 0x00007fd2ea6fec63 in select () from /lib/libc.so.6 # 2012-09-06T01:42:59 #1 0x00000000009b0cdc in os_thread_sleep (tm=tm@entry=5000000) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0thread.c:259 # 2012-09-06T01:42:59 #2 0x0000000000890c78 in srv_LRU_dump_restore_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2933 # 2012-09-06T01:42:59 #3 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #4 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 16 (Thread 0x7fd2e84e5700 (LWP 2634)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207 # 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609 # 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=3) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458 # 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453 # 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419 # 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=0, pass@entry=140543208226584, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779 # 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942 # 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59 # 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc028678, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252 # 2012-09-06T01:42:59 #10 0x00000000008ddf3d in btr_cur_open_at_index_side_func (from_left=from_left@entry=1, index=index@entry=0x7fd2d001b728, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc028678, file=file@entry=0xb160b8 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic", line=line@entry=519, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:941 # 2012-09-06T01:42:59 #11 0x0000000000876aee in btr_pcur_open_at_index_side (mtr=0x7fd2e84e2dc0, do_init=0, pcur=0x7fd2cc028678, latch_mode=1, index=0x7fd2d001b728, from_left=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:518 # 2012-09-06T01:42:59 #12 row_search_for_mysql (buf=0x7fd2cc0239c8 "\377", mode=1, prebuilt=0x7fd2cc028608, match_mode=0, direction=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:3976 # 2012-09-06T01:42:59 #13 0x00000000008349ae in ha_innobase::index_read (this=0x7fd2cc023da0, buf=0x7fd2cc0239c8 "\377", key_ptr=, key_len=0, find_flag=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:6665 # 2012-09-06T01:42:59 #14 0x0000000000832071 in ha_innobase::index_first (this=0x7fd2cc023da0, buf=0x7fd2cc0239c8 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7015 # 2012-09-06T01:42:59 #15 0x000000000083f703 in ha_innobase::rnd_next (this=0x7fd2cc023da0, buf=0x7fd2cc0239c8 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7112 # 2012-09-06T01:42:59 #16 0x0000000000a18506 in ha_partition::rnd_next (this=0x7fd2cc023460, buf=0x7fd2cc0239c8 "\377") at /media/Vone/percona-server/origin/Percona-Server/sql/ha_partition.cc:4032 # 2012-09-06T01:42:59 #17 0x0000000000777233 in rr_sequential (info=0x7fd2e84e3590) at /media/Vone/percona-server/origin/Percona-Server/sql/records.cc:455 # 2012-09-06T01:42:59 #18 0x0000000000613f85 in mysql_update (thd=0x2d8b6c0, table_list=0x7fd2c4004e38, fields=..., values=..., conds=0x0, order_num=, order=0x0, limit=18446744073709551615, handle_duplicates=DUP_ERROR, ignore=false, found_return=0x7fd2e84e38f0, updated_return=0x7fd2e84e38f8) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_update.cc:653 # 2012-09-06T01:42:59 #19 0x000000000059785c in mysql_execute_command (thd=thd@entry=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2940 # 2012-09-06T01:42:59 #20 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2d8b6c0, rawbuf=, length=73, parser_state=parser_state@entry=0x7fd2e84e4760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811 # 2012-09-06T01:42:59 #21 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2d8b6c0, packet=packet@entry=0x2d8ee01 "UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 7", packet_length=packet_length@entry=73) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061 # 2012-09-06T01:42:59 #22 0x000000000059e75f in do_command (thd=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788 # 2012-09-06T01:42:59 #23 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484 # 2012-09-06T01:42:59 #24 0x0000000000648157 in handle_one_connection (arg=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391 # 2012-09-06T01:42:59 #25 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #26 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 15 (Thread 0x7fd2d935a700 (LWP 2602)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=1, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=7, message1=message1@entry=0x7fd2d9359e68, message2=message2@entry=0x7fd2d9359e60, type=type@entry=0x7fd2d9359e58, space_id=space_id@entry=0x7fd2d9359e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=7) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 14 (Thread 0x7fd2e8567700 (LWP 2615)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09e37d in read () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x0000000000a3abda in vio_read (vio=0x2d7fb10, buf=0x2d7fd00 "\a", size=4) at /media/Vone/percona-server/origin/Percona-Server/vio/viosocket.c:56 # 2012-09-06T01:42:59 #2 0x000000000051f33f in my_real_read (net=net@entry=0x2d7c4c0, complen=complen@entry=0x7fd2e8566d88) at /media/Vone/percona-server/origin/Percona-Server/sql/net_serv.cc:832 # 2012-09-06T01:42:59 #3 0x000000000051fd41 in my_net_read (net=0x2d7c4c0) at /media/Vone/percona-server/origin/Percona-Server/sql/net_serv.cc:1029 # 2012-09-06T01:42:59 #4 0x000000000059e5f5 in do_command (thd=0x2d7c290) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:734 # 2012-09-06T01:42:59 #5 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d7c290) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484 # 2012-09-06T01:42:59 #6 0x0000000000648157 in handle_one_connection (arg=0x2d7c290) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391 # 2012-09-06T01:42:59 #7 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #8 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 13 (Thread 0x7fd2d74ab700 (LWP 2606)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09bcc1 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000009b04cd in os_cond_wait_timed (abstime=0x7fd2d74aadc0, mutex=0x27fa460, cond=0x27fa498) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:171 # 2012-09-06T01:42:59 #2 os_event_wait_time_low (event=0x27fa460, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:716 # 2012-09-06T01:42:59 #3 0x000000000089001d in srv_lock_timeout_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2674 # 2012-09-06T01:42:59 #4 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #5 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 12 (Thread 0x7fd2ea408740 (LWP 2593)): # 2012-09-06T01:42:59 #0 0x00007fd2ea6fcebd in poll () from /lib/libc.so.6 # 2012-09-06T01:42:59 #1 0x000000000051c231 in poll (__timeout=-1, __nfds=2, __fds=0x7fffe26b9850) at /usr/include/bits/poll2.h:41 # 2012-09-06T01:42:59 #2 handle_connections_sockets () at /media/Vone/percona-server/origin/Percona-Server/sql/mysqld.cc:5361 # 2012-09-06T01:42:59 #3 0x000000000051e100 in mysqld_main (argc=47, argv=0x254a298) at /media/Vone/percona-server/origin/Percona-Server/sql/mysqld.cc:4800 # 2012-09-06T01:42:59 #4 0x0000000000513975 in main (argc=, argv=) at /media/Vone/percona-server/origin/Percona-Server/sql/main.cc:25 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 11 (Thread 0x7fd2d54a7700 (LWP 2610)): # 2012-09-06T01:42:59 #0 0x00007fd2ea6fec63 in select () from /lib/libc.so.6 # 2012-09-06T01:42:59 #1 0x00000000009b0cdc in os_thread_sleep (tm=tm@entry=899000) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0thread.c:259 # 2012-09-06T01:42:59 #2 0x0000000000891a36 in srv_master_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:3298 # 2012-09-06T01:42:59 #3 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #4 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 10 (Thread 0x7fd2d4ca6700 (LWP 2611)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207 # 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609 # 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458 # 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453 # 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419 # 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779 # 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942 # 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59 # 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=page@entry=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x2cc1068, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252 # 2012-09-06T01:42:59 #10 0x00000000008dd45d in btr_cur_search_to_nth_level (index=0x7fd2d001b728, level=level@entry=0, tuple=tuple@entry=0x2cc15a8, mode=mode@entry=4, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x2cc1068, has_search_latch=has_search_latch@entry=0, file=file@entry=0xb1ac58 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0row.c", line=line@entry=716, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:756 # 2012-09-06T01:42:59 #11 0x000000000086c136 in btr_pcur_open_func (mtr=0x7fd2d4ca4b40, line=716, file=0xb1ac58 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0row.c", cursor=0x2cc1068, latch_mode=1, mode=4, tuple=0x2cc15a8, index=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:438 # 2012-09-06T01:42:59 #12 row_search_on_row_ref (pcur=pcur@entry=0x2cc1068, mode=mode@entry=1, table=0x7fd2d0026858, ref=0x2cc15a8, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0row.c:716 # 2012-09-06T01:42:59 #13 0x00000000009e9a56 in row_purge_reposition_pcur (mode=mode@entry=1, node=node@entry=0x2cc1008, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:101 # 2012-09-06T01:42:59 #14 0x00000000009ea871 in row_purge_poss_sec (node=node@entry=0x2cc1008, index=index@entry=0x7fd2d001aa48, entry=entry@entry=0x7fd2c0000ca8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:256 # 2012-09-06T01:42:59 #15 0x00000000009eab2c in row_purge_remove_sec_if_poss_leaf (node=node@entry=0x2cc1008, index=index@entry=0x7fd2d001aa48, entry=entry@entry=0x7fd2c0000ca8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:383 # 2012-09-06T01:42:59 #16 0x00000000009eb305 in row_purge_remove_sec_if_poss (entry=0x7fd2c0000ca8, index=0x7fd2d001aa48, node=0x2cc1008) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:433 # 2012-09-06T01:42:59 #17 row_purge_upd_exist_or_extern_func (thr=thr@entry=0x2cc0e50, node=node@entry=0x2cc1008) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:541 # 2012-09-06T01:42:59 #18 0x00000000009ec69c in row_purge (thr=0x2cc0e50, node=0x2cc1008) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:770 # 2012-09-06T01:42:59 #19 row_purge_step (thr=thr@entry=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:805 # 2012-09-06T01:42:59 #20 0x00000000009d468f in que_thr_step (thr=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/que/que0que.c:1259 # 2012-09-06T01:42:59 #21 que_run_threads_low (thr=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/que/que0que.c:1319 # 2012-09-06T01:42:59 #22 que_run_threads (thr=thr@entry=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/que/que0que.c:1356 # 2012-09-06T01:42:59 #23 0x00000000008a2a42 in trx_purge (limit=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/trx/trx0purge.c:1195 # 2012-09-06T01:42:59 #24 0x0000000000893d26 in srv_purge_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:3967 # 2012-09-06T01:42:59 #25 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #26 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 9 (Thread 0x7fd2dbb5f700 (LWP 2597)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=2, message1=message1@entry=0x7fd2dbb5ee68, message2=message2@entry=0x7fd2dbb5ee60, type=type@entry=0x7fd2dbb5ee58, space_id=space_id@entry=0x7fd2dbb5ee50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=2) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 8 (Thread 0x7fd2d64a9700 (LWP 2608)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09bcc1 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000009b04cd in os_cond_wait_timed (abstime=0x7fd2d64a8e10, mutex=0x27fa4e0, cond=0x27fa518) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:171 # 2012-09-06T01:42:59 #2 os_event_wait_time_low (event=0x27fa4e0, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:716 # 2012-09-06T01:42:59 #3 0x000000000088fd14 in srv_monitor_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2524 # 2012-09-06T01:42:59 #4 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #5 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 7 (Thread 0x7fd2d8358700 (LWP 2604)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=3, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=9, message1=message1@entry=0x7fd2d8357e68, message2=message2@entry=0x7fd2d8357e60, type=type@entry=0x7fd2d8357e58, space_id=space_id@entry=0x7fd2d8357e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=9) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 6 (Thread 0x7fd2d8b59700 (LWP 2603)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=2, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=8, message1=message1@entry=0x7fd2d8b58e68, message2=message2@entry=0x7fd2d8b58e60, type=type@entry=0x7fd2d8b58e58, space_id=space_id@entry=0x7fd2d8b58e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 5 (Thread 0x7fd2d9b5b700 (LWP 2601)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=6, message1=message1@entry=0x7fd2d9b5ae68, message2=message2@entry=0x7fd2d9b5ae60, type=type@entry=0x7fd2d9b5ae58, space_id=space_id@entry=0x7fd2d9b5ae50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=6) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 4 (Thread 0x7fd2da35c700 (LWP 2600)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=3, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=5, message1=message1@entry=0x7fd2da35be68, message2=message2@entry=0x7fd2da35be60, type=type@entry=0x7fd2da35be58, space_id=space_id@entry=0x7fd2da35be50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=5) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 3 (Thread 0x7fd2db35e700 (LWP 2598)): # 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1 # 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=1, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646 # 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=3, message1=message1@entry=0x7fd2db35de68, message2=message2@entry=0x7fd2db35de60, type=type@entry=0x7fd2db35de58, space_id=space_id@entry=0x7fd2db35de50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791 # 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=3) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535 # 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483 # 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 2 (Thread 0x7fd2e84a4700 (LWP 2636)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207 # 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609 # 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458 # 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453 # 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419 # 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=0, pass@entry=140543006899528, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779 # 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942 # 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59 # 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2b800e978, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252 # 2012-09-06T01:42:59 #10 0x00000000008ddf3d in btr_cur_open_at_index_side_func (from_left=from_left@entry=1, index=index@entry=0x7fd2d001b728, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2b800e978, file=file@entry=0xb160b8 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic", line=line@entry=519, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:941 # 2012-09-06T01:42:59 #11 0x0000000000876aee in btr_pcur_open_at_index_side (mtr=0x7fd2e84a1dc0, do_init=0, pcur=0x7fd2b800e978, latch_mode=1, index=0x7fd2d001b728, from_left=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:518 # 2012-09-06T01:42:59 #12 row_search_for_mysql (buf=0x7fd2b8008678 "\377", mode=1, prebuilt=0x7fd2b800e908, match_mode=0, direction=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:3976 # 2012-09-06T01:42:59 #13 0x00000000008349ae in ha_innobase::index_read (this=0x7fd2b800a0a0, buf=0x7fd2b8008678 "\377", key_ptr=, key_len=0, find_flag=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:6665 # 2012-09-06T01:42:59 #14 0x0000000000832071 in ha_innobase::index_first (this=0x7fd2b800a0a0, buf=0x7fd2b8008678 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7015 # 2012-09-06T01:42:59 #15 0x000000000083f703 in ha_innobase::rnd_next (this=0x7fd2b800a0a0, buf=0x7fd2b8008678 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7112 # 2012-09-06T01:42:59 #16 0x0000000000a18506 in ha_partition::rnd_next (this=0x7fd2b8009b50, buf=0x7fd2b8008678 "\377") at /media/Vone/percona-server/origin/Percona-Server/sql/ha_partition.cc:4032 # 2012-09-06T01:42:59 #17 0x0000000000777233 in rr_sequential (info=0x7fd2e84a2590) at /media/Vone/percona-server/origin/Percona-Server/sql/records.cc:455 # 2012-09-06T01:42:59 #18 0x0000000000613f85 in mysql_update (thd=0x2d92f40, table_list=0x7fd2b8004e38, fields=..., values=..., conds=0x0, order_num=, order=0x0, limit=18446744073709551615, handle_duplicates=DUP_ERROR, ignore=false, found_return=0x7fd2e84a28f0, updated_return=0x7fd2e84a28f8) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_update.cc:653 # 2012-09-06T01:42:59 #19 0x000000000059785c in mysql_execute_command (thd=thd@entry=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2940 # 2012-09-06T01:42:59 #20 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2d92f40, rawbuf=, length=73, parser_state=parser_state@entry=0x7fd2e84a3760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811 # 2012-09-06T01:42:59 #21 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2d92f40, packet=packet@entry=0x2d96681 "UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 1", packet_length=packet_length@entry=73) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061 # 2012-09-06T01:42:59 #22 0x000000000059e75f in do_command (thd=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788 # 2012-09-06T01:42:59 #23 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484 # 2012-09-06T01:42:59 #24 0x0000000000648157 in handle_one_connection (arg=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391 # 2012-09-06T01:42:59 #25 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #26 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 # 2012-09-06T01:42:59 Thread 1 (Thread 0x7fd2e85a8700 (LWP 2630)): # 2012-09-06T01:42:59 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #1 0x00000000007d6c58 in my_write_core (sig=6) at /media/Vone/percona-server/origin/Percona-Server/mysys/stacktrace.c:433 # 2012-09-06T01:42:59 #2 0x00000000006ad21c in handle_fatal_signal (sig=6) at /media/Vone/percona-server/origin/Percona-Server/sql/signal_handler.cc:249 # 2012-09-06T01:42:59 #3 # 2012-09-06T01:42:59 #4 0x00007fd2ea655fa5 in raise () from /lib/libc.so.6 # 2012-09-06T01:42:59 #5 0x00007fd2ea657428 in abort () from /lib/libc.so.6 # 2012-09-06T01:42:59 #6 0x0000000000986ff5 in rec_offs_validate (rec=rec@entry=0x7fd2e3d904ec "\200", index=index@entry=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/rem0rec.ic:973 # 2012-09-06T01:42:59 #7 0x000000000098a017 in lock_rec_queue_validate (block=block@entry=0x7fd2dbfe9740, rec=rec@entry=0x7fd2e3d904ec "\200", index=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4756 # 2012-09-06T01:42:59 #8 0x0000000000990a5a in lock_rec_validate_page (block=0x7fd2dbfe9740) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4965 # 2012-09-06T01:42:59 #9 lock_rec_block_validate (page_no=, space=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5056 # 2012-09-06T01:42:59 #10 lock_validate () at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5109 # 2012-09-06T01:42:59 #11 lock_print_info_all_transactions (file=file@entry=0x294d5a0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4580 # 2012-09-06T01:42:59 #12 0x000000000088f112 in srv_printf_innodb_monitor (file=0x294d5a0, nowait=0, trx_start=0x7fd2e85a6780, trx_end=0x7fd2e85a6788) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2243 # 2012-09-06T01:42:59 #13 0x0000000000839db7 in innodb_show_status (stat_print=0x6ad7aa , thd=0x2ced230, hton=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10374 # 2012-09-06T01:42:59 #14 innobase_show_status (hton=, thd=0x2ced230, stat_print=0x6ad7aa , stat_type=) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10614 # 2012-09-06T01:42:59 #15 0x00000000006b5876 in ha_show_status (thd=0x2ced230, db_type=0x254adb0, stat=HA_ENGINE_STATUS) at /media/Vone/percona-server/origin/Percona-Server/sql/handler.cc:5066 # 2012-09-06T01:42:59 #16 0x0000000000595bfa in mysql_execute_command (thd=thd@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2480 # 2012-09-06T01:42:59 #17 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2ced230, rawbuf=, length=25, parser_state=parser_state@entry=0x7fd2e85a7760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811 # 2012-09-06T01:42:59 #18 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2ced230, packet=packet@entry=0x2d78151 "", packet_length=packet_length@entry=25) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061 # 2012-09-06T01:42:59 #19 0x000000000059e75f in do_command (thd=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788 # 2012-09-06T01:42:59 #20 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484 # 2012-09-06T01:42:59 #21 0x0000000000648157 in handle_one_connection (arg=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391 # 2012-09-06T01:42:59 #22 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0 # 2012-09-06T01:42:59 #23 0x00007fd2ea70504d in clone () from /lib/libc.so.6 # 2012-09-06T01:42:59 Test completed with failure status STATUS_SERVER_CRASHED (101) # 2012-09-06T01:42:59 gentest.pl exited with exit status STATUS_SERVER_CRASHED (101) 2012-09-06T01:42:59 [2527] runall.pl will exit with exit status STATUS_SERVER_CRASHED (101) It is crashing on ha_show_status which may be related to bug 1043620 and/or to the issue http://bugs.mysql.com/bug.php?id=60682 (which also deals with ha_show_status associated locking, also used as test case in https://bugs.launchpad.net/percona-server/+bug/1040735)