lockfile-create hangs inside lxc containers (potential buffer overflow?)

Thanks for submitting this bug, James.

I tried to reproduce it by simply installing ntp in two containers, but failed.

However, my hunch is that it is due to bug 925024.

The fix for that bug had been queued in the precise kernel source, but is not yet (I think) in the archive.

Could you disable the apparmor profile using

    sudo apparmor_parser -R /etc/apparmor.d/usr.bin.lxc-start

and see if you can still reproduce this?

I played around with this a bit and derived that it is due to the length of the hostname (at least in my case). With a hostname of length 40 I can reproduce this error. Setting the hostname to <= 37 produces the correct results.

I also do not believe that this bug is a duplicate and apparmor is only the culprit in that it is finding a potential overflow somewhere in the ntp code

If I turn apparmour off, I can no longer effectively install through juju if the package includes NTP. I'll work to reproduce with a minimal environment.

I'm the same has @breu. the juju charm I was using was making the hostname of my system 40 characters. Reducing the charm name length solved the problem. I think this bug should be re-opened and fixed in NTP or in LXC or wherever the problem lies.

I've unmarked this as a duplicate and marked it confirmed as there are two people reporting this as an overflow in ntp.

I will mark this bug as affecting NTP and temporarily assign it to jjohansen, with apologies, for advice.

I decided not to assign to jjohansen, only subscribe him :)

If noone else gets to the bottom of this before I clear some things off my plate, I will see if I can get to the bottom of it. I would assign myself to it, but dont' want to stop someone else from jumping in if they have time.

Brue and davidpbritton,

could you please attach the charm you were using?

Simply running lockfile-create, killing it quickly, and re-running it in a container doesn't reproduce it for me. Can you please tell me which juju charm to use to reproduce this? Can you still reproduce this today?

[Expired for lxc (Ubuntu) because there has been no activity for 60 days.]

[Expired for ntp (Ubuntu) because there has been no activity for 60 days.]

I accidentally reproduced this, using:
username project64198
environment democluster
service hbase-master

project64198-democluster-hbase-master-2 has 40 characters. After I destroyed the environment and recreated under the name demo, everything was running fine.

Sebastian,

I'm sorry, could you please give us more specifically the steps you took? What was that username for - is that the logged-in username on the host? is the environment the environment name in .juju/environments.yaml, and service the name of the charm you invoked?

Sure. First, I ssh'd to project64198@deneb, a (metal) server of mine. There prepared for local deployment:

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:juju/pkgs
sudo apt-get update && sudo apt-get install juju
sudo apt-get install lxc apt-cacher-ng libzookeeper-java zookeeper

Now I ran juju bootstrap and got a .juju/environments.yaml, which I modiified:

environments:
  democluster:
    type: local
    control-bucket: juju-c81e728d9d4c2f636f067f89cc14862c
    admin-secret: eccbc87e4b5ce2fe28308fd9f2a7baf3
    default-series: precise
    juju-origin: ppa
    data-dir: /home/project64198/democluster

Also, I ran ssh-keygen -t rsa and mkdir democluster, and prepared for hbase by writing this config.yaml:

hadoop-master:
    hbase: True
hadoop-workers:
    hbase: True
thrift:
    monitoring_port: 9090

After another juju bootstrap I deployed my cloud:

juju deploy zookeeper
juju deploy --config config.yaml hadoop hadoop-master
juju deploy --config config.yaml hadoop hadoop-workers
juju deploy hbase hbase-master
juju deploy hbase hbase-regions
juju add-relation hadoop-master:namenode hadoop-workers:datanode
juju add-relation hbase-master zookeeper
juju add-relation hbase-regions zookeeper
juju add-relation hadoop-master:namenode hbase-master:namenode
juju add-relation hadoop-master:namenode hbase-regions:namenode
juju add-relation hbase-master:master hbase-regions:regionserver

It was at this point that hbase-master would stick in agent-state: pending, public-address: null
So I ran juju destroy-environment, changed these lines in .juju/environments.yaml:

  demo:
    data-dir: /home/project64198/demo

After an mkdir demo, juju bootstrap and all the deploys and add-relations, my cloud was up.

@Sebastian,

what you are seeing is likely unrelated to this particular bug. However if I simply do something like:

juju deploy --repository=~/myrepo local:precise/ovs-lxc lxc-5678901234567890123456789012345678901

and very quickly

juju debug-log

then I see:

LXCError: lxc-start: node name '' is too long
lxc-start: failed to read configuration file
Traceback (most recent call last):

Could you confirm whether that is what you also see? And if so, file it as a bug against lxc and juju?

@jamespage,

do you still see this bug?

Never mind - simply doing

sudo hostname host567890123456789012345678901234567890
sudo apt-get install lxc

in precise reproduced this, giving me

 * Starting NTP server ntpd *** glibc detected *** lockfile-create: free(): invalid next size (fast): 0x0000000000d750a0 ***

Note I get the same thing in a kvm-based cloud instance.

Marking this bug as not affecting lxc.

I have reproduced this on precise and raring.

The problem is with string handling in liblockfile's lockfile_create_save_tmplock(). I'll start work on getting a debdiff prepared.

Here's debdiff for Raring. It passes the [Test Case] that I added to the bug description, as well as other manual testing such as a creating a lockfile on a system with a 1 character long hostname.

The debdiff also fixes bug #1011477. See that bug for info and test case.

I just uploaded your patch to raring, and sent it on to Debian bug http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=677225

This bug was fixed in the package liblockfile - 1.09-5ubuntu1

---------------
liblockfile (1.09-5ubuntu1) raring; urgency=low

  * debian/patches/fix-buffer-overflows.patch: Fix buffer overflows when
    building strings
    - Protect against overflows caused by long hostnames (LP: #941968)
    - Protect against overflows caused by large PID numbers (LP: #1011477)
 -- Tyler Hicks <email address hidden> Wed, 09 Jan 2013 12:23:07 -0800

this may be a dup of #971314

or do I have to type bug#971314 or LP #971314 or what

Hitting this on precise trying to deploy a more recent Ceph (which apparently uses liblockfile) on an openstack cloud configured to use its default hostname scheme, eg: server-e5476307-a3c3-4703-b28c-84fedce7078.novalocal

The raring nomination can be removed/rejected. The patch that fixes the issue was released in raring circa liblockfile 1.09-5ubuntu1.

Sponsored precise upload. Unsubscribing sponsors.

Hello James, or anyone else affected,

Accepted liblockfile into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/liblockfile/1.09-3ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello James, or anyone else affected,

Accepted liblockfile into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/liblockfile/1.09-4ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Came here by way of the ntp restart failing due to liblockfile errors if the hostname is too long. The 1.09-3ubuntu0.1 proposed package fixes the error for me, thanks!

I tested this and the bug still exists where lockfile-create segfaults with a long hostname. The problem here is the 23 characters allowed for the system name is still not sufficient. gethostname() can return a hostname up to 256 characters long.

on precise with liblockfile-bin 1.09-3ubuntu0.1 installed:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
root@areallylonghostnamethatshouldbreakeverything:~# lockfile-create /var/lock/ntpdate
*** glibc detected *** lockfile-create: free(): invalid next size (fast): 0x000000000128f0a0 ***
Segmentation fault (core dumped)

from the patch:
~~~~~~~~~~~~
#define TMPLOCKSTR ".lk"
#define TMPLOCKSTRSZ strlen(TMPLOCKSTR)
+#define TMPLOCKPIDSZ 5
#define TMPLOCKTIMESZ 1
#define TMPLOCKSYSNAMESZ 23
#define TMPLOCKFILENAMESZ (TMPLOCKSTRSZ + TMPLOCKPIDSZ + \
     TMPLOCKTIMESZ + TMPLOCKSYSNAMESZ)

TMPLOCKSYSNAMESZ needs to be much larger than 23. This should actually be the same as the size of sysname which in this case is 256.

Can we please get this released in a precise SRU?

Joseph, I cannot reproduce this with the version in precise-proposed. Are you sure that you installed 1.09-3ubuntu0.1? Also the maximum the kernel stores as a hostname is 64 characters, so there should never be 256 characters returned.

This bug was fixed in the package liblockfile - 1.09-3ubuntu0.1

---------------
liblockfile (1.09-3ubuntu0.1) precise-proposed; urgency=low

  * debian/patches/fix-buffer-overflows.patch: Fix buffer overflows when
    building strings
    - Protect against overflows caused by long hostnames (LP: #941968)
    - Protect against overflows caused by large PID numbers (LP: #1011477)
 -- Adam Gandelman <email address hidden> Thu, 20 Jun 2013 12:37:10 -0700

Phillipp

I am, and it is still not fixed

root@breu-prec-jenkins-32675-api:~# apt-cache policy liblockfile1
liblockfile1:
  Installed: 1.09-3
  Candidate: 1.09-3ubuntu0.1
  Version table:
     1.09-3ubuntu0.1 0
        500 http://archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages
 *** 1.09-3 0
        500 http://archive.ubuntu.com/ubuntu/ precise/main amd64 Packages
        100 /var/lib/dpkg/status
root@breu-prec-jenkins-32675-api:~# hostname
breu-prec-jenkins-32675-api
root@breu-prec-jenkins-32675-api:~# hostname areallylonghostnamethatshouldbreakeverything
root@breu-prec-jenkins-32675-api:~# hostname
areallylonghostnamethatshouldbreakeverything
root@breu-prec-jenkins-32675-api:~# lockfile-create /var/lock/123date
*** glibc detected *** lockfile-create: free(): invalid next size (fast): 0x00000000024bc0a0 ***
Segmentation fault (core dumped)

this is also on precise

never mind - my co-worker let me down and this totally works. my bad.

quantal has seen the end of its life and is no longer receiving any updates. Marking the quantal task for this ticket as "Won't Fix".