On Thu, 2010-12-23 at 12:40 +0000, ingo wrote:
> @Clint:
>
> I did test your proposal in Maverick.
>
> Before editing the stop scripts:
>
> fgrep "stop on runlevel" /etc/init/*.conf
> /etc/init/acpid.conf:stop on runlevel [!2345]
> /etc/init/anacron.conf:stop on runlevel [!2345]
> /etc/init/apport.conf:stop on runlevel [!2345]
> /etc/init/atd.conf:stop on runlevel [!2345]
> /etc/init/cron.conf:stop on runlevel [!2345]
> /etc/init/cups.conf:stop on runlevel [016]
> /etc/init/dbus.conf:stop on runlevel [06]
> /etc/init/failsafe-x.conf:stop on runlevel [06]
> /etc/init/gdm.conf:stop on runlevel [016]
> /etc/init/irqbalance.conf:stop on runlevel [!2345]
> /etc/init/mountall-shell.conf:stop on runlevel [06]
> /etc/init/rc.conf:stop on runlevel [!$RUNLEVEL]
> /etc/init/rcS.conf:stop on runlevel [!S]
> /etc/init/rc-sysinit.conf:stop on runlevel
> /etc/init/rsyslog.conf:stop on runlevel [06]
> /etc/init/tty1.conf:stop on runlevel [!2345]
> /etc/init/tty2.conf:stop on runlevel [!23]
> /etc/init/tty3.conf:stop on runlevel [!23]
> /etc/init/tty4.conf:stop on runlevel [!23]
> /etc/init/tty5.conf:stop on runlevel [!23]
> /etc/init/tty6.conf:stop on runlevel [!23]
> /etc/init/udev.conf:stop on runlevel [06]
> /etc/init/ufw.conf:stop on runlevel [!023456]
>
> After editing the stop scripts:
>
> fgrep "stop on starting" /etc/init/*.conf
> /etc/init/cups.conf:stop on starting rc RUNLEVEL=[016]
> /etc/init/dbus.conf:stop on starting rc RUNLEVEL=[06]
> /etc/init/failsafe-x.conf:stop on starting rc RUNLEVEL=[06]
> /etc/init/gdm.conf:stop on starting rc RUNLEVEL=[016]
> /etc/init/mountall.conf:stop on starting rcS
> /etc/init/mountall-shell.conf:stop on starting rc RUNLEVEL=[06]
> /etc/init/rsyslog.conf:stop on starting rc RUNLEVEL=[06]
> /etc/init/udev.conf:stop on starting rc RUNLEVEL=[06]
>
> Then execute
>
> apt-get install --reinstall libc6
> and reboot:
>
> I still get the 8 orphaned inodes as reported already.
>
> Did I miss to change the other scrips as well like this?
> stop on runlevel [!2345] -> stop on stop on starting rc RUNLEVEL=[016]
>
Yes, and some of those are probably the most likely to have libc open.
If doing the same to all of the !2345's does not fix the corruption, can
you do:
apt-get install --reinstall libc6
lsof -n |grep deleted
initctl list
And paste or upload the output of that here?
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/688541
>
> Title:
> race condition on shutdown (leads to corrupted fs)
>
> Status in “mysql-5.1” package in Ubuntu:
> Triaged
> Status in “sysvinit” package in Ubuntu:
> Triaged
>
> Bug description:
> I'm using mysql-server-5.1 on a 10.04 LTS installation.
> The mysql db is around 27GB and on a separate partition mounted as /var/lib/mysql.
>
> On shutdown I get the following error message:
>
> Checking for running unattended-upgrades: * Asking all remaining processes to terminate...
> [80G
> [74G[ OK ]
> * All processes ended within 1 seconds....
> [80G
> [74G[ OK ]
> * Deconfiguring network interfaces...
> [80G
> [74G[ OK ]
> * Deactivating swap...
> [80G
> [74G[ OK ]
> * Unmounting local filesystems...
> [80G umount2: Device or resource busy
> umount: /var/lib/mysql: device is busy.
> (In some cases useful info about processes that use
> the device is found by lsof(8) or fuser(1))
> umount2: Device or resource busy
> umount2: Device or resource busy
> umount: /tmp: device is busy.
> (In some cases useful info about processes that use
> the device is found by lsof(8) or fuser(1))
> umount2: Device or resource busy
> [74G[
> [31mfail
> [39;49m]
> mount: / is busy
> * Will now restart
> [ 3369.429751] Restarting system.
>
>
> On the next reboot the file system is corrupt and need to be fsck-ed.
>
> I think the problem is, that mysql uses an upstart job (/etc/init/mysql.conf) and has
> stop on runlevel [016]
>
> The rc.conf job is also triggered on runlevel 0 and 6, so they basically run at the same time.As
>
> When /etc/rc0.d/S20sendsigs is run, it deliberatly does not wait or kill any upstart jobs.
>
> As my mysqld process takes some time to shutdown, S40umountfs and S60umountroot are run before the mysqld has quit.
>
> Leading to the fs not being properly unmounted. It is event possible that mysqld is forcefully killed by halt in S90halt if it hasn't stopped by then.
>
> This is a serious issue, as it can (and will) lead to data loss.
>
> Other upstart jobs, like rsyslog.conf, use the same "stop on runlevel [016]" stanza, so they are probably affected too.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 10.10
> Package: mysql-server-5.1 5.1.49-1ubuntu8.1
> Uname: Linux 2.6.32-5-686 i686
> NonfreeKernelModules: michael_mic arc4 ecb lib80211_crypt_tkip aes_i586 aes_generic lib80211_crypt_ccmp sco bnep rfcomm l2cap binfmt_misc acpi_cpufreq ppdev lp cpufreq_userspace cpufreq_stats vboxnetadp cpufreq_powersave vboxnetflt cpufreq_conservative vboxdrv fuse pcmcia snd_intel8x0m snd_intel8x0 snd_ac97_codec btusb bluetooth rfkill ac97_bus yenta_socket ipw2200 snd_pcm 8139too firewire_ohci snd_seq 8139cp firewire_core sg uhci_hcd snd_timer rsrc_nonstatic libipw snd_seq_device pcmcia_core crc_itu_t parport_pc smsc_ircc2 ehci_hcd mii joydev lib80211 sr_mod parport i2c_i801 irda snd usbcore wbsd soundcore shpchp mmc_core pcspkr rng_core cdrom psmouse container crc_ccitt snd_page_alloc serio_raw pci_hotplug ac battery nls_base processor evdev ppp_generic slhc loop autofs4 ext4 mbcache jbd2 crc16 dm_mod sd_mod crc_t10dif radeon ttm ata_generic drm_kms_helper ata_piix drm i2c_algo_bit libata video thermal i2c_core scsi_mod output thermal_sys button
> Architecture: i386
> Date: Fri Dec 10 13:41:52 2010
> ProcEnviron:
> PATH=(custom, no user)
> LANG=de_DE.utf8
> SHELL=/bin/bash
> SourcePackage: mysql-5.1
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/mysql-5.1/+bug/688541/+subscribe
On Thu, 2010-12-23 at 12:40 +0000, ingo wrote: acpid.conf: stop on runlevel [!2345] anacron. conf:stop on runlevel [!2345] apport. conf:stop on runlevel [!2345] atd.conf: stop on runlevel [!2345] cron.conf: stop on runlevel [!2345] cups.conf: stop on runlevel [016] dbus.conf: stop on runlevel [06] failsafe- x.conf: stop on runlevel [06] gdm.conf: stop on runlevel [016] irqbalance. conf:stop on runlevel [!2345] mountall- shell.conf: stop on runlevel [06] rc.conf: stop on runlevel [!$RUNLEVEL] rcS.conf: stop on runlevel [!S] rc-sysinit. conf:stop on runlevel rsyslog. conf:stop on runlevel [06] tty1.conf: stop on runlevel [!2345] tty2.conf: stop on runlevel [!23] tty3.conf: stop on runlevel [!23] tty4.conf: stop on runlevel [!23] tty5.conf: stop on runlevel [!23] tty6.conf: stop on runlevel [!23] udev.conf: stop on runlevel [06] ufw.conf: stop on runlevel [!023456] cups.conf: stop on starting rc RUNLEVEL=[016] dbus.conf: stop on starting rc RUNLEVEL=[06] failsafe- x.conf: stop on starting rc RUNLEVEL=[06] gdm.conf: stop on starting rc RUNLEVEL=[016] mountall. conf:stop on starting rcS mountall- shell.conf: stop on starting rc RUNLEVEL=[06] rsyslog. conf:stop on starting rc RUNLEVEL=[06] udev.conf: stop on starting rc RUNLEVEL=[06]
> @Clint:
>
> I did test your proposal in Maverick.
>
> Before editing the stop scripts:
>
> fgrep "stop on runlevel" /etc/init/*.conf
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
>
> After editing the stop scripts:
>
> fgrep "stop on starting" /etc/init/*.conf
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
> /etc/init/
>
> Then execute
>
> apt-get install --reinstall libc6
> and reboot:
>
> I still get the 8 orphaned inodes as reported already.
>
> Did I miss to change the other scrips as well like this?
> stop on runlevel [!2345] -> stop on stop on starting rc RUNLEVEL=[016]
>
Yes, and some of those are probably the most likely to have libc open.
If doing the same to all of the !2345's does not fix the corruption, can
you do:
apt-get install --reinstall libc6
lsof -n |grep deleted
initctl list
And paste or upload the output of that here?
> -- /bugs.launchpad .net/bugs/ 688541 upgrades: * Asking all remaining processes to terminate... mysql.conf) and has d/S20sendsigs is run, it deliberatly does not wait or kill any upstart jobs. dules: michael_mic arc4 ecb lib80211_crypt_tkip aes_i586 aes_generic lib80211_crypt_ccmp sco bnep rfcomm l2cap binfmt_misc acpi_cpufreq ppdev lp cpufreq_userspace cpufreq_stats vboxnetadp cpufreq_powersave vboxnetflt cpufreq_ conservative vboxdrv fuse pcmcia snd_intel8x0m snd_intel8x0 snd_ac97_codec btusb bluetooth rfkill ac97_bus yenta_socket ipw2200 snd_pcm 8139too firewire_ohci snd_seq 8139cp firewire_core sg uhci_hcd snd_timer rsrc_nonstatic libipw snd_seq_device pcmcia_core crc_itu_t parport_pc smsc_ircc2 ehci_hcd mii joydev lib80211 sr_mod parport i2c_i801 irda snd usbcore wbsd soundcore shpchp mmc_core pcspkr rng_core cdrom psmouse container crc_ccitt snd_page_alloc serio_raw pci_hotplug ac battery nls_base processor evdev ppp_generic slhc loop autofs4 ext4 mbcache jbd2 crc16 dm_mod sd_mod crc_t10dif radeon ttm ata_generic drm_kms_helper ata_piix drm i2c_algo_bit libata video thermal i2c_core scsi_mod output thermal_sys button /bugs.launchpad .net/ubuntu/ +source/ mysql-5. 1/+bug/ 688541/ +subscribe
> You received this bug notification because you are a direct subscriber
> of the bug.
> https:/
>
> Title:
> race condition on shutdown (leads to corrupted fs)
>
> Status in “mysql-5.1” package in Ubuntu:
> Triaged
> Status in “sysvinit” package in Ubuntu:
> Triaged
>
> Bug description:
> I'm using mysql-server-5.1 on a 10.04 LTS installation.
> The mysql db is around 27GB and on a separate partition mounted as /var/lib/mysql.
>
> On shutdown I get the following error message:
>
> Checking for running unattended-
> [80G
> [74G[ OK ]
> * All processes ended within 1 seconds....
> [80G
> [74G[ OK ]
> * Deconfiguring network interfaces...
> [80G
> [74G[ OK ]
> * Deactivating swap...
> [80G
> [74G[ OK ]
> * Unmounting local filesystems...
> [80G umount2: Device or resource busy
> umount: /var/lib/mysql: device is busy.
> (In some cases useful info about processes that use
> the device is found by lsof(8) or fuser(1))
> umount2: Device or resource busy
> umount2: Device or resource busy
> umount: /tmp: device is busy.
> (In some cases useful info about processes that use
> the device is found by lsof(8) or fuser(1))
> umount2: Device or resource busy
> [74G[
> [31mfail
> [39;49m]
> mount: / is busy
> * Will now restart
> [ 3369.429751] Restarting system.
>
>
> On the next reboot the file system is corrupt and need to be fsck-ed.
>
> I think the problem is, that mysql uses an upstart job (/etc/init/
> stop on runlevel [016]
>
> The rc.conf job is also triggered on runlevel 0 and 6, so they basically run at the same time.As
>
> When /etc/rc0.
>
> As my mysqld process takes some time to shutdown, S40umountfs and S60umountroot are run before the mysqld has quit.
>
> Leading to the fs not being properly unmounted. It is event possible that mysqld is forcefully killed by halt in S90halt if it hasn't stopped by then.
>
> This is a serious issue, as it can (and will) lead to data loss.
>
> Other upstart jobs, like rsyslog.conf, use the same "stop on runlevel [016]" stanza, so they are probably affected too.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 10.10
> Package: mysql-server-5.1 5.1.49-1ubuntu8.1
> Uname: Linux 2.6.32-5-686 i686
> NonfreeKernelMo
> Architecture: i386
> Date: Fri Dec 10 13:41:52 2010
> ProcEnviron:
> PATH=(custom, no user)
> LANG=de_DE.utf8
> SHELL=/bin/bash
> SourcePackage: mysql-5.1
>
> To unsubscribe from this bug, go to:
> https:/