Maintainer scripts mishandle /var/cache/bind permissions

Thank you for taking the time to report this bug and helping to make Ubuntu better.

I've not been able to reproduce this when upgrading from Lucid to Precise (1:9.7.0.dfsg.P1-1ubuntu0.8 to 1:9.8.1.dfsg.P1-4ubuntu0.4). /var/cache/bind had the correct (775) permissions. If I remove its contents and change the permissions to 755, then I do see the 100% CPU usage and the error in the log file that you've reported. But what I don't see is how to get the permissions to the erroneous 755 in the first place - simply installing bind9 in Lucid and upgrading to Precise doesn't seem to do it.

Are you sure that the permissions weren't already wrong due to a local misconfiguration before you upgraded?

Marking as Incomplete for now. If you manage to figure out how to reproduce this problem, please comment and change the bug status back to New.

The server concerns was automatically installed from a CD-ROM built from Ubuntu sources and (in respect of bind) it has only had automatic updates run on it. I am very confident it was not operator error.

It was upgraded with 'do-release-upgrade'.

I can tell you I am not the only person experiencing this. See for instance:
  http://ubuntuforums.org/showthread.php?t=1971471
  https://bugs.launchpad.net/ubuntu/+source/bind9/+bug/1038199 (same root cause I'm guessing)

I would have thought that given >1 people are seeing this, a chmod in the postinst file would do no harm.

With as many users as we have, a common misconfiguration can lead to a number of reports. I'd have expected dozens of reports or more by now if this were a systemic upgrade problem. Also note that one of the reports you linked to has root group ownership too, which is inconsistent with a single root cause.

I've looked at the ways the bind9 maintainer scripts call chmod, and don't see how a group write permission could get lost:

$ grep chmod /var/lib/dpkg/info/*bind*
/var/lib/dpkg/info/bind9.postinst: chmod 775 /var/lib/bind
/var/lib/dpkg/info/bind9.postinst: chmod g+s /etc/bind
/var/lib/dpkg/info/bind9.postinst: chmod g+r /etc/bind/rndc.key /etc/bind/named.conf* || true
/var/lib/dpkg/info/bind9.postinst: chmod g+rwx /var/run/named /var/cache/bind

I've checked the maintainer scripts this way across Hardy, Lucid and Precise. It looks to me that the existing postinst intentionally avoids doing the chmod except in a particular circumstance which I presume is for upgrading from a specific previous version (presumably prior to Hardy). I'd like to understand the root cause before I'm comfortable pushing to change this, and there is a trivial workaround for those affected.

So steps to reproduce the problem would really help!

Well I'm pretty sure the problem is this. I've just gone to another (unconnected) Lucid box, and:

root@extility-developers:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 10.04.4 LTS
Release: 10.04
Codename: lucid
root@extility-developers:~# ls -ln /etc/bind/rndc.key
-rw-r----- 1 103 108 77 2012-06-14 14:23 /etc/bind/rndc.key

See rndc.key is owned by UID 103, which is not equal to 0. So the Precise postinst script does not do the chmod.

You may not have received reports because bind actually works, just uses high CPU.

On Wed, Dec 05, 2012 at 05:48:18PM -0000, Alex Bligh wrote:
> See rndc.key is owned by UID 103, which is not equal to 0. So the
> Precise postinst script does not do the chmod.

This is what I would expect, because the permissions on /var/cache/bind
should already be correct, and maintainer scripts generally try not to
interfere with anything the administrator might have done.

OK so my working hypothesis is this. On Lucid /var/cache/bind is created simply by virtue of it being a directory within the package (see the bind9.list file). The group write permission is added by the postinst. If the Lucid package was installed, then removed, then installed again, the following happens:

1. the first install would create /var/cache/bind with whatever ownership is in the package, and also /etc/bind/rndc.key with root ownership. The postinst thens runs and fixes the group write permission on /var/cache/bind.

2. the removal would delete /var/cache/bind as it is not a conffile, but not /etc/bind/rndc.key

3. the second install would create /var/cache/bind again with (possibly) the wrong permissions, and the postinst script would not fix it.

This probably doesn't go wrong in Lucid because nothing writes to the cache directory and/or bind survives without the cache. It's certainly empty here on our Lucid boxes pre upgrade to Precise. But the Precise upgrade requires to write there, and then dies.

The above would happen (AFAICT) if *ANY* version ever released of the Lucid bind9.deb had broken permissions, as subsequent upgrades would not fix it.

The problem with only fixing permissions if some rather random file in /etc/ is owned by root is it is inherently fragile. Is there any reason why the bind cache directory should ever not be writeable by the group that owns it?

To follow this up, the .deb at least on Lucid does NOT have the write permission set.

amb@nimrod-ubuntu:~/bind-test$ dpkg -c bind9_9.7.0.dfsg.P1-1ubuntu0.8_amd64.deb | fgrep cache
drwxr-xr-x root/root 0 2012-10-09 14:13 ./var/cache/
drwxr-xr-x root/root 0 2012-10-09 14:13 ./var/cache/bind/

I've tried this on a pristine Precise box and it doesn't go wrong because Precise does not remove /var/cache/bind as it is populated (unlike on at least some Lucid installs). However, if I manually remove the cache directory, it does go wrong:

root@adamant:~# dpkg --list bind9
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
ii bind9 1:9.8.1.dfsg.P Internet Domain Name Server
root@adamant:~# ls -lnd /var/cache/bind /etc/bind/rndc.key
-rw-r----- 1 103 108 77 Dec 3 20:56 /etc/bind/rndc.key
drwxrwxr-x 2 0 108 4096 Dec 4 21:00 /var/cache/bind
root@adamant:~# aptitude remove bind9
The following packages will be REMOVED:
  bind9
0 packages upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
Need to get 0 B of archives. After unpacking 963 kB will be freed.
(Reading database ... 47095 files and directories currently installed.)
Removing bind9 ...
 * Stopping domain name service... bind9
waiting for pid 859 to die
   ...done.
Processing triggers for ufw ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Processing triggers for man-db ...

root@adamant:~# ls -lnd /var/cache/bind /etc/bind/rndc.key
-rw-r----- 1 103 108 77 Dec 3 20:56 /etc/bind/rndc.key
drwxrwxr-x 2 0 108 4096 Dec 5 19:13 /var/cache/bind
root@adamant:~# ls -la /var/cache/bind
total 16
drwxrwxr-x 2 root bind 4096 Dec 5 19:13 .
drwxr-xr-x 8 root root 4096 Dec 3 20:54 ..
-rw-r--r-- 1 bind bind 698 Dec 4 21:00 managed-keys.bind
-rw-r--r-- 1 bind bind 512 Dec 4 21:00 managed-keys.bind.jnl
root@adamant:~# rm -rf /var/cache/bind
root@adamant:~# aptitude install bind9
The following NEW packages will be installed:
  bind9
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 343 kB of archives. After unpacking 963 kB will be used.
Get: 1 http://gb.archive.ubuntu.com/ubuntu/ precise-updates/main bind9 amd64 1:9.8.1.dfsg.P1-4ubuntu0.4 [343 kB]
Fetched 343 kB in 0s (621 kB/s)
Preconfiguring packages ...
Selecting previously unselected package bind9.
(Reading database ... 47062 files and directories currently installed.)
Unpacking bind9 (from .../bind9_1%3a9.8.1.dfsg.P1-4ubuntu0.4_amd64.deb) ...
Processing triggers for man-db ...
Processing triggers for ureadahead ...
Processing triggers for ufw ...
Setting up bind9 (1:9.8.1.dfsg.P1-4ubuntu0.4) ...
 * Starting domain name service... bind9
   ...done.

root@adamant:~# ls -lnd /var/cache/bind /etc/bind/rndc.key
-rw-r----- 1 103 108 77 Dec 3 20:56 /etc/bind/rndc.key
drwxr-xr-x 2 0 0 4096 Oct 9 14:06 /v...

Thanks for your insight Alex. I've managed to reproduce this now, with the following steps:

On Lucid:
 1. sudo apt-get install bind9
 2. sudo apt-get remove bind9
 # this removes /var/cache/bind but leaves /etc/bind9/rndc.key
 3. sudo apt-get install bind9
 # Now the postinst doesn't fix /var/cache/bind, but on Lucid nobody will notice this problem
 4. sudo do-release-upgrade
 # bind now uses /var/cache/bind/managed-keys.bind and the problem occurs

After the upgrade to Precise, bind9 is in the situation you described (permissions on /var/cache/bind wrong), with 100% CPU consumption.

I couldn't reproduce the problem directly on Precise because on package removal /var/cache/bind/managed-keys.bind is left behind and so /var/cache/bind never gets removed (I think this is a separate bug in itself).

It seems likely to me that this issue will affect Debian also, so next I will test this and file a bug report in Debian as needed, so that we can coordinate a fix.

Robie,

No problem - I'm just glad I wasn't imagining it.

I agree the 100% CPU problem can't be reproduced on precise.

To be honest I don't quite understand why /var/cache/bind isn't in /var/run (given it's a cache) but I may be wrong about that.

Alex

There are two problems here. First is that the bind9.postinst fails to set permissions correctly in some circumstances, and second that without the permissions set correctly, you get 100% CPU usage.

The 100% CPU usage problem is already in bug 1038199, so we can track that there. That leaves this bug to track the postinst /var/cache/bind permissions problem.