lxc scripts break when user has PYTHONPATH set

Bug #1130809 reported by Nicola Heald
72
This bug affects 12 people
Affects Status Importance Assigned to Milestone
pyjuju
Fix Committed
Critical
Kapil Thangavelu
juju (Ubuntu)
Fix Released
Critical
Stephanie
lxc (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

The various lxc scripts installed in /usr/bin are Python 3 only. If the user has PYTHONPATH or other similar environment variables set, these scripts can incorrectly inherit details from the user's Python 2 setup when run.

This can result in either a crash as per the initial report, or a traceback over syntax differences such as the following in juju:

Traceback (most recent call last):
   File "/usr/lib/python2.7/dist-packages/juju/agents/machine.py", line 97, in watch_service_units
     yield self.unit_deployer.start_service_unit(unit_name)
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1069, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python2.7/dist-packages/juju/unit/deploy.py", line 91, in start_service_unit
     running = yield deployment.is_running()
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1069, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python2.7/dist-packages/juju/machine/unit.py", line 301, in is_running
     prefix=self.container.container_name)
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1069, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python2.7/dist-packages/juju/lib/lxc/__init__.py", line 147, in get_containers
     _, output = yield deferToThread(_cmd, ["lxc-ls"])
   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 172, in _worker
     result = context.call(ctx, function, *args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python2.7/dist-packages/juju/lib/lxc/__init__.py", line 48, in _cmd
     raise LXCError(stdout_data)
 LXCError: Fatal Python error: Py_Initialize: Unable to get the locale encoding
   File "/usr/lib/python2.7/encodings/__init__.py", line 123
     raise CodecRegistryError,\
                             ^
 SyntaxError: invalid syntax

WORKAROUND:
Add "-Es" to /usr/bin/lxc-ls (after /usr/bin/python3 ) so that those environment variables are ignored.

Related branches

Revision history for this message
Nicola Heald (notnownikki) wrote :
Revision history for this message
Apport retracing service (apport) wrote :

StacktraceTop:
 Py_FatalError (msg=msg@entry=0x81d3504 "Py_Initialize: Unable to get the locale encoding") at ../Python/pythonrun.c:2365
 initfsencoding.isra.4.21966 () at ../Python/pythonrun.c:873
 _Py_InitializeEx_Private (install_sigs=install_sigs@entry=1, install_importlib=install_importlib@entry=1) at ../Python/pythonrun.c:373
 Py_InitializeEx (install_sigs=install_sigs@entry=1) at ../Python/pythonrun.c:401
 Py_Initialize () at ../Python/pythonrun.c:407

Revision history for this message
Apport retracing service (apport) wrote : Stacktrace.txt
Revision history for this message
Apport retracing service (apport) wrote : StacktraceSource.txt
Revision history for this message
Apport retracing service (apport) wrote : ThreadStacktrace.txt
Changed in lxc (Ubuntu):
importance: Undecided → Medium
summary: - lxc-ls crashed with SIGABRT in raise()
+ lxc-ls crashed with SIGABRT in Py_FatalError()
tags: removed: need-i386-retrace
Revision history for this message
Stéphane Graber (stgraber) wrote : Re: lxc-ls crashed with SIGABRT in Py_FatalError()

I've been looking into this bug, here's the full environment as passed to lxc-ls:
SUDO_GID=200512
MAIL=/var/mail/root
USER=root
LANGUAGE=en_CA:en
HOME=/root
SUDO_UID=201105
LOGNAME=root
USERNAME=root
TERM=unknown
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
JUJU_HOME=/tmp/juju/stgraber-sample
LANG=en_CA.UTF-8
JUJU_ORIGIN=distro
SUDO_COMMAND=/usr/bin/python -m juju.agents.machine --logfile /tmp/juju/stgraber-sample/machine-agent.log --zookeeper-servers 10.0.3.1:40010 --juju-directory /tmp/juju/stgraber-sample --machine-id 0 --session-file /var/run/juju/stgraber-sample-machine-agent.zksession --pidfile /tmp/juju/stgraber-sample/machine-agent.pid
SHELL=/bin/bash
JUJU_ZOOKEEPER=10.0.3.1:40010
JUJU_UNIT_NS=stgraber-sample
JUJU_SERIES=precise
SUDO_USER=stgraber
JUJU_PUBLIC_KEY=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6yjMe+wWsH9xgor39al4p+W22D7Io8+n0ZahNEWyTG0rLZDYMQryrXUDYJRvAbiCVfTB4e7waLTDoaK5sSqoa0GoexYCIgFduOBHZfjWHu9HcpIeoq5Bc+XmOYQaoFjL4YEc2GAP//8CEmGLCqC/+rO36YbBPjd0176pAJQ7R2Bym+4k2/VvA4Plw439IdElGMgK2ge+5xc0qq7iph+ITCriR1eYCorA6T2j97ufddvQO/fmOh9+HoQ4Y+i6mKSPNyO9m87+KImwF1XX/ocQRBO9eYkqWM992ZqXG4H62r+NoRNwW4R9c5nF6rUc8x1StTpp3L0A0ybIYjRuV3m+J stgraber@lantea
JUJU_MACHINE_ID=0
PWD=/home/stgraber
PYTHONPATH=/usr/bin:/usr/lib/python2.7:/usr/lib/python2.7/plat-i386-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/PILcompat:/usr/lib/python2.7/dist-packages/gst-0.10:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/ubuntu-sso-client:/usr/lib/python2.7/dist-packages/ubuntuone-client:/usr/lib/python2.7/dist-packages/ubuntuone-control-panel:/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol

You'll notice that JuJu overrides PYTHONPATH with 2.7 paths when lxc-ls is a python3 script.
This in turn causes the python interpreter to fail as can be seen below:
stgraber@lantea:~$ export PYTHONPATH=/usr/bin:/usr/lib/python2.7:/usr/lib/python2.7/plat-i386-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/PILcompat:/usr/lib/python2.7/dist-packages/gst-0.10:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/ubuntu-sso-client:/usr/lib/python2.7/dist-packages/ubuntuone-client:/usr/lib/python2.7/dist-packages/ubuntuone-control-panel:/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol
stgraber@lantea:~$ python3
Fatal Python error: Py_Initialize: Unable to get the locale encoding
  File "/usr/lib/python2.7/encodings/__init__.py", line 123
    raise CodecRegistryError,\
                            ^
SyntaxError: invalid syntax
Aborted (core dumped)
stgraber@lantea:~$

So the bug is actually in juju which doesn't pass a clean environment to the scripts it's calling and triggers this bug.

information type: Private → Public
affects: lxc (Ubuntu) → juju (Ubuntu)
Revision history for this message
Stéphane Graber (stgraber) wrote :

A temporary workaround would be to add "-Es" to /usr/bin/lxc-ls (after /usr/bin/python3 ) so that those environment variables are ignored.

Changed in juju (Ubuntu):
status: New → Triaged
importance: Medium → Critical
Revision history for this message
rowez (info-rowez) wrote :
Download full text (6.0 KiB)

Upgraded to 13.04 (raring ..).

twistd 12.3.0 is using /usr/bin/python 2.7.3 (storage-server.log)

Have no PYTHONPATH.

Only PATH is present.

And /usr/bin/python is a symlink to python2.7.

And /usr/bin/python3 is a symlink to python3.3.

Only the machine is running in juju status.

The service mysql is pending and pending. ......

The -Es is not present in /usr/bin/lxc-ls

and got this error in machine-agent.log:

2013-03-21 22:06:43,565: juju.agents.machine@INFO: Machine agent started id:0
2013-03-21 22:06:51,683: juju.agents.machine@DEBUG: Units changed old:set([]) new:set(['mysql/0'])
2013-03-21 22:06:51,684: juju.agents.machine@DEBUG: Starting service unit: mysql/0 ...
2013-03-21 22:06:51,706: unit.deploy@DEBUG: Downloading charm cs:precise/mysql-15 to /home/ron/juju/ron-rowez/charms
2013-03-21 22:06:51,733: unit.deploy@DEBUG: Using <juju.machine.unit.UnitContainerDeployment object at 0x1efd4d0> for mysql/0 in /home/ron/juju/ron-rowez
2013-03-21 22:06:52,131: twisted@ERROR: Fatal Python error: Py_Initialize: Unable to get the locale encoding
2013-03-21 22:06:52,131: twisted@ERROR: File "/usr/lib/python2.7/encodings/__init__.py", line 123
2013-03-21 22:06:52,131: twisted@ERROR: raise CodecRegistryError,\
2013-03-21 22:06:52,131: twisted@ERROR: ^
2013-03-21 22:06:52,131: twisted@ERROR: SyntaxError: invalid syntax
2013-03-21 22:06:52,132: twisted@ERROR:
2013-03-21 22:06:52,133: juju.agents.machine@ERROR: Error starting unit: mysql/0
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/juju/agents/machine.py", line 97, in watch_service_units
    yield self.unit_deployer.start_service_unit(unit_name)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1069, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/juju/unit/deploy.py", line 91, in start_service_unit
    running = yield deployment.is_running()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1069, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/juju/machine/unit.py", line 301, in is_running
    prefix=self.container.container_name)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1069, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/juju/lib/lxc/__init__.py", line 147, in get_containers
    _, output = yield deferToThread(_cmd, ["lxc-ls"])
  File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 172, in _worker
    result = context.call(ctx, function, *args, **kwargs)
  File "...

Read more...

Curtis Hovey (sinzui)
description: updated
Martin Packman (gz)
summary: - lxc-ls crashed with SIGABRT in Py_FatalError()
+ lxc scripts break when user has PYTHONPATH
summary: - lxc scripts break when user has PYTHONPATH
+ lxc scripts break when user has PYTHONPATH set
Martin Packman (gz)
description: updated
Revision history for this message
Martin Packman (gz) wrote :

I'm unconvinced this is a problem with the use of lxc in juju as opposed to a general issue with the lxc scripts.

Changed in juju:
status: New → Incomplete
Revision history for this message
Curtis Hovey (sinzui) wrote :

What exactly would the lxc scripts be doing wrong? Since the work around involves the extraordinary option of dropping the environment, I think what ever is forcing a py2 path is being naughty.

Revision history for this message
Nicola Heald (notnownikki) wrote : Re: [Bug 1130809] Re: lxc scripts break when user has PYTHONPATH set

On 08/04/13 18:54, Curtis Hovey wrote:
> What exactly would the lxc scripts be doing wrong? Since the work around
> involves the extraordinary option of dropping the environment, I think
> what ever is forcing a py2 path is being naughty.
I'd agree here. A python2 app calling a python3 app should ensure that
the environment isn't polluted with python2 paths.

Revision history for this message
Stéphane Graber (stgraber) wrote :

To make thinking this is an LXC bug as oppose to a JuJu one because lxc-ls is python and so can be affected by environment variables makes as much sense as blaming a C program for having a different behaviour when your code change PATH or LD_PRELOAD.

Marking this invalid for LXC (I'm not opposed to dh_python3 changing the shebang to -Es but I won't do this pre-emptively in lxc).

Changed in lxc (Ubuntu):
status: New → Invalid
Martin Packman (gz)
Changed in juju (Ubuntu):
assignee: nobody → Martin Packman (gz)
status: Triaged → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package juju - 0.7-0ubuntu1

---------------
juju (0.7-0ubuntu1) raring; urgency=low

  [ Clint Byrum ]
  * d/p/fix-tests-do-not-use-etc-lsb-release: Dropped, patch applied upstream.
  * d/p/maas-tag-conversion.patch: Dropped, patch applied upstream.
  * d/juju.docs: Remove examples as they have been dropped upstream.

  [ Mark Mims ]
  * Add postinst and prerm scripts using update-alternatives to install into
    a versioned location, enabling co-installability with the go juju port.

  [ Martin Packman ]
  * New upstream release. (lp: #1167921)
  * d/p/workaround-lxc-python-env.patch: Workaround regression with local
    provider failing with SyntaxError on running lxc scripts when user has
    Python 3 specific environment variables set. (lp: #1130809)
 -- Martin Packman <email address hidden> Wed, 10 Apr 2013 13:57:21 +0000

Changed in juju (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Alan Pope 🍺🐧🐱 🦄 (popey) wrote :

Excellent, working well, thanks.

Changed in juju:
status: Incomplete → Fix Committed
milestone: none → 0.7.1
assignee: nobody → Kapil Thangavelu (hazmat)
Curtis Hovey (sinzui)
Changed in juju:
importance: Undecided → Critical
Changed in juju (Ubuntu):
assignee: Martin Packman (gz) → Stephanie (districtmellow-1904)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.