apache crashes with *** glibc detected *** /usr/sbin/apache2: double free or corruption (fasttop): 0x00007f8b5c8fdac0 ***

Bug #651134 reported by Paul Vaughan
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
apache2 (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

Binary package hint: apache2

Expected: no errors.

Actual: Maybe crashes isn't quite the right phrase, as the server itself is still servings pages, however we've seen the below error and backtrace regularly since we upgraded our server and once Apache stopped responding. We're seeing a general trend in increasing numbers of processes.

LAMP: Ubuntu 10.04.1 LTS, Apache/2.2.14 (Ubuntu) (Server built: Apr 13 2010 20:21:26), PHP 5.3.2-1ubuntu4.2 with Suhosin-Patch (cli) (built: May 13 2010 20:03:45)

# apt-cache policy apache2
apache2:
  Installed: 2.2.14-5ubuntu8
  Candidate: 2.2.14-5ubuntu8.2
  Version table:
     2.2.14-5ubuntu8.2 0
        500 http://gb.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        500 http://security.ubuntu.com/ubuntu/ lucid-security/main Packages
 *** 2.2.14-5ubuntu8 0
        500 http://gb.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status
# apt-cache policy php5
php5:
  Installed: 5.3.2-1ubuntu4.2
  Candidate: 5.3.2-1ubuntu4.5
  Version table:
     5.3.2-1ubuntu4.5 0
        500 http://gb.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        500 http://security.ubuntu.com/ubuntu/ lucid-security/main Packages
 *** 5.3.2-1ubuntu4.2 0
        100 /var/lib/dpkg/status
     5.3.2-1ubuntu4 0
        500 http://gb.archive.ubuntu.com/ubuntu/ lucid/main Packages

MySQL server is not installed.

Error message as found in /var/log/apache2/error.log (attached).
---
Architecture: amd64
DistroRelease: Ubuntu 10.04
InstallationMedia: Ubuntu-Server 10.04 LTS "Lucid Lynx" - Release amd64 (20100427)
Package: apache2 2.2.14-5ubuntu8
PackageArchitecture: amd64
ProcEnviron:
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.32-21.32-server 2.6.32.11+drm33.2
Tags: lucid
Uname: Linux 2.6.32-21-server x86_64
UserGroups: admin

Revision history for this message
Paul Vaughan (paulieboo) wrote :
Revision history for this message
James Page (james-page) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command, as it will automatically gather debugging information, in a terminal:

apport-collect 651134

When reporting bugs in the future please use apport, using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https://wiki.ubuntu.com/ReportingBugs.

Changed in apache2 (Ubuntu):
status: New → Incomplete
Revision history for this message
Paul Vaughan (paulieboo) wrote :

Have run apport-collect on the server via a remote ssh session but despite trying to sign in with my email address and password (good text interface by the way), the continue text is inactive. I can get a forgotten password, create a new account or cancel (all work) but I cannot continue.

Have never used this before so it's possible I'm doing something wrong, but I can log in just fine to Launchpad via a browser on my desktop machine.

Revision history for this message
Paul Vaughan (paulieboo) wrote : Dependencies.txt

apport information

tags: added: apport-collected
description: updated
Revision history for this message
Paul Vaughan (paulieboo) wrote :

Got around the issue by using the solution found here:

https://bugs.launchpad.net/launchpad-foundations/+bug/556927

apport-collect report sent successfully.

Revision history for this message
James Page (james-page) wrote :

Please can you try to obtain a backtrace following the instructions at http://wiki.ubuntu.com/DebuggingProgramCrash and upload the backtrace (as an attachment) to the bug report. This will greatly help us in tracking down your problem.

Thanks

Revision history for this message
Paul Vaughan (paulieboo) wrote :

Followed the instructions on DebuggingProgramCrash as far as:

$ sudo apt-get install apache2.2-bin-dbgsym

Received:

The following packages have unmet dependencies.
  apache2.2-bin-dbgsym: Depends: apache2.2-bin (= 2.2.14-5ubuntu8.3) but 2.2.14-5ubuntu8 is to be installed
E: Broken packages

apt-get updated and upgraded, and tried again:

The following packages have unmet dependencies.
  apache2.2-bin-dbgsym: Depends: apache2.2-bin (= 2.2.14-5ubuntu8.3) but 2.2.14-5ubuntu8.2 is to be installed
E: Broken packages

It does not appear possible to update any further than this, and now I am stuck. (Sorry.)

Revision history for this message
Paul Vaughan (paulieboo) wrote :

I have just realised that having updated/upgraded, I may have 1. solved the problem (you never know) and 2. I may have now changed the version numbers of key apps and libraries as reported earlier.

Revision history for this message
James Page (james-page) wrote :

Assuming that you continue to see this issue now that you have upgrade apache to 2.2.14-5ubuntu8.2 you can install the correct version of the -dbgsym package by running the following command:

    sudo apt-get install apache2.2-bin-dbgsym=2.2.14-5ubuntu8.2

This will ensure that the version of -dbgsym matches the installed binaries.

If the upgrade appears to have resolved the issue please let us know

Thanks

Revision history for this message
Paul Vaughan (paulieboo) wrote :

Thanks for the above, it worked successfully.

I will keep a close eye on Apache today (a normal day's use, basically) and report back.

Regards.

Revision history for this message
Paul Vaughan (paulieboo) wrote :

After approximately 2 days of normal use, the error has not reoccurred (whereas normally we'd see it approximately 3-5 times in a 24 hour period).

I guess we can consider this issue 'gone'.

Cheers.

Revision history for this message
James Page (james-page) wrote :

I think so; I'll close this bug report down.

Revision history for this message
James Page (james-page) wrote :

This bug report is being closed due to your last comment regarding this being fixed with an update.

For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status.

Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.

Changed in apache2 (Ubuntu):
status: Incomplete → Invalid
Revision history for this message
Paul Vaughan (paulieboo) wrote :

I spoke too soon. 21 of the 'glibc detected' reports in the Apache error log. Please see attached. I'll work on the backtrace this afternoon.

James Page (james-page)
Changed in apache2 (Ubuntu):
status: Invalid → Incomplete
Revision history for this message
Paul Vaughan (paulieboo) wrote :

I have gdb running on the server, but as I have no idea what causes the error I'll just watch Apache's error log until I see it print out the backtrace and memory map.

Revision history for this message
Christine G. (ctrobbia) wrote :

I am not sure if this is even related. I have received in the past the above error verbatim. But the last few weeks I've been getting various errors that start out: *** glibc detected *** /usr/sbin/apache2:

usually what this results in is apache delivering a blank page (until I restart apache) or like the above - it segfaults but still delivers pages however I have an increase in processes until it's out of control and I have to kill them and restart apache.

Attached is my error log for most recent error (a snippet).

I can't tell if it's really apache or php (found this: http://bugs.php.net/bug.php?id=49209 but it's not for my version of php but I do see that error too)

Here's some other info:

apache2:
  Installed: 2.2.14-5ubuntu8
  Candidate: 2.2.14-5ubuntu8
  Version table:
 *** 2.2.14-5ubuntu8 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status

php5:
  Installed: 5.3.2-1ubuntu4.2
  Candidate: 5.3.2-1ubuntu4.2
  Version table:
 *** 5.3.2-1ubuntu4.2 0
        500 http://security.ubuntu.com/ubuntu/ lucid-updates/main Packages
        100 /var/lib/dpkg/status
     5.3.2-1ubuntu4 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages

I apologize, ahead of time, if this is in the wrong place. Just let me know and I will correct ASAP.

Revision history for this message
Paul Vaughan (paulieboo) wrote :

Hi Christine.

Since upgrading Apache to 2.2.14-5ubuntu8.2 we haven't seen this issue resurface in over a week. I've also noticed that 2.2.14-5ubuntu8.3 is available so may tentatively upgrade to that. I hope that's of some use.

Paul.

Revision history for this message
Christine G. (ctrobbia) wrote :

Thanks, Paul. I will upgrade now and see how it goes. I appreciate your reply!

Revision history for this message
Christine G. (ctrobbia) wrote :
Download full text (31.7 KiB)

Still happens.
[Sun Oct 31 11:14:47 2010] [notice] Apache/2.2.14 (Ubuntu) mod_fcgid/2.3.4 PHP/5.3.2-1ubuntu4.5 with Suhosin-Patch mod_ssl/2.2.14 OpenSSL/0.9.8k configured -- resuming normal operations
*** glibc detected *** /usr/sbin/apache2: double free or corruption (!prev): 0xbfd3fa18 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0xb76cd591]
/lib/tls/i686/cmov/libc.so.6(+0x6cde8)[0xb76cede8]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0xb76d1ecd]
/usr/lib/apache2/modules/libphp5.so(+0x297d0c)[0xb70abd0c]
/usr/lib/apache2/modules/libphp5.so(+0x19dda9)[0xb6fb1da9]
/usr/lib/apache2/modules/libphp5.so(zend_error+0x542)[0xb7104a22]
/usr/lib/apache2/modules/libphp5.so(+0x34e89b)[0xb716289b]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/apache2/modules/libphp5.so(zend_call_function+0x841)[0xb70f9f31]
/usr/lib/apache2/modules/libphp5.so(call_user_function_ex+0x64)[0xb70fab74]
/usr/lib/apache2/modules/libphp5.so(php_pcre_replace_impl+0xa19)[0xb6ea9569]
/usr/lib/apache2/modules/libphp5.so(php_pcre_replace+0x65)[0xb6ea9a35]
/usr/lib/apache2/modules/libphp5.so(+0x95b40)[0xb6ea9b40]
/usr/lib/apache2/modules/libphp5.so(+0x96267)[0xb6eaa267]
/usr/lib/apache2/modules/libphp5.so(execute_internal+0x4b)[0xb712b73b]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x15ea8)[0xb4d9dea8]
/usr/lib/apache2/modules/libphp5.so(+0x343398)[0xb7157398]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/apache2/modules/libphp5.so(zend_call_function+0x841)[0xb70f9f31]
/usr/lib/apache2/modules/libphp5.so(+0x2136b5)[0xb70276b5]
/usr/lib/apache2/modules/libphp5.so(execute_internal+0x4b)[0xb712b73b]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x15ea8)[0xb4d9dea8]
/usr/lib/apache2/modules/libphp5.so(+0x343398)[0xb7157398]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/apache2/modules/libphp5.so(+0x343086)[0xb7157086]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/apache2/modules/libphp5.so(+0x343086)[0xb7157086]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/apache2/modules/libphp5.so(+0x343086)[0xb7157086]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/apache2/modules/libphp5.so(+0x343086)[0xb7157086]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/apache2/modules/libphp5.so(zend_call_function+0x841)[0xb70f9f31]
/usr/lib/apache2/modules/libphp5.so(+0x2136b5)[0xb70276b5]
/usr/lib/apache2/modules/libphp5.so(execute_internal+0x4b)[0xb712b73b]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x15ea8)[0xb4d9dea8]
/usr/lib/apache2/modules/libphp5.so(+0x343398)[0xb7157398]
/usr/lib/apache2/modules/libphp5.so(execute+0x1ce)[0xb712e0ce]
/usr/lib/php5/20090626+lfs/suhosin.so(+0x16314)[0xb4d9e314]
/usr/lib/...

Revision history for this message
James Page (james-page) wrote :

Christine

Please could you raise your issue in a separate bug report; if you could also follow the same debugging process in #6 and try to grab a backtrace with debug enabled it will help with the diagnostic process.

Thanks

James

Revision history for this message
James Page (james-page) wrote :

Paul

Are you still experiencing this issue since upgrading? Its a bit unclear from your last post.

James

Revision history for this message
Paul Vaughan (paulieboo) wrote :

Hi James.

At this time, with the latest build of Apache as mentioned in my last post, I'm not seeing this issue any more. I'll open a new issue if I do see it again.

Cheers,

Paul.

Revision history for this message
James Page (james-page) wrote :

This bug report is being closed due to your last comment regarding this being fixed with an update.

For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status.

Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.

Changed in apache2 (Ubuntu):
status: Incomplete → Invalid
Revision history for this message
tinodj (gjorgjioski) wrote :
Download full text (4.0 KiB)

Hi,

I have got this error two times on my server today.

Frist time:
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ee0e0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef590 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ef700 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ef7f0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738efd70 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ef890 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738efc50 ***
*** glibc detected *** /usr/sbin/apache2: free(): invalid pointer: 0x00007fff738ef380 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738efb30 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef450 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ef070 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738eecb0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef2b0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef450 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738efb20 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ef190 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef5b0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738efcd0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef630 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef610 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ef0f0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738ef5b0 ***
*** glibc detected *** /usr/sbin/apache2: free(): invalid pointer: 0x00007fff738f0140 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff738efcd0 ***
*** glibc detected *** /usr/sbin/apache2: munmap_chunk(): invalid pointer: 0x00007fff738ef620 ***

Second time:
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff3a86fdb0 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff3a870e80 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff3a870630 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff3a870370 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff3a86fe30 ***
*** glibc detected *** /usr/sbin/apache2: double free or corruption (out): 0x00007fff3a870500 ***
*** glibc detected *** /usr/sbin/apache2: free(): invalid pointer: 0x00007fff3a870ae0 ***
**...

Read more...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.