get_swift_codename is subject to apt race conditions

Bug #1561087 reported by Francis Ginther
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Swift Proxy Charm
Triaged
Low
Unassigned
OpenStack Swift Storage Charm
Triaged
Low
Unassigned
swift-proxy (Juju Charms Collection)
Invalid
Undecided
Unassigned

Bug Description

Hit this traceback with cs:~openstack-charmers-next/trusty/swift-proxy-43:

2016-03-23 06:36:11 INFO swift-storage-relation-changed E: Problem renaming the file /var/
cache/apt/pkgcache.bin.CWnMSY to /var/cache/apt/pkgcache.bin - rename (2: No such file or
directory)
2016-03-23 06:36:11 INFO swift-storage-relation-changed W: You may want to run apt-get upd
ate to correct these problems
2016-03-23 06:36:11 INFO swift-storage-relation-changed Traceback (most recent call last):
2016-03-23 06:36:11 INFO swift-storage-relation-changed File "/var/lib/juju/agents/unit-swift-proxy-0/charm/hooks/swift-storage-relation-changed", line 97, in <module>
2016-03-23 06:36:11 INFO swift-storage-relation-changed CONFIGS = register_configs()
2016-03-23 06:36:11 INFO swift-storage-relation-changed File "/var/lib/juju/agents/unit-swift-proxy-0/charm/hooks/lib/swift_utils.py", line 296, in register_configs
2016-03-23 06:36:11 INFO swift-storage-relation-changed release = get_os_codename_package('swift-proxy', fatal=False) \
2016-03-23 06:36:11 INFO swift-storage-relation-changed File "/var/lib/juju/agents/unit-swift-proxy-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 303, in get_os_codename_package
2016-03-23 06:36:11 INFO swift-storage-relation-changed return get_swift_codename(vers)
2016-03-23 06:36:11 INFO swift-storage-relation-changed File "/var/lib/juju/agents/unit-swift-proxy-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 252, in get_swift_codename
2016-03-23 06:36:11 INFO swift-storage-relation-changed ret = subprocess.check_output(['apt-cache', 'policy', 'swift'])
2016-03-23 06:36:11 INFO swift-storage-relation-changed File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
2016-03-23 06:36:11 INFO swift-storage-relation-changed raise CalledProcessError(retcode, cmd, output=output)
de, cmd, output=output)
2016-03-23 06:36:11 INFO swift-storage-relation-changed subprocess.CalledProcessError: Command '['apt-cache', 'policy', 'swift']' returned non-zero exit status 100
2016-03-23 06:36:11 ERROR juju.worker.uniter.operation runhook.go:107 hook "swift-storage-relation-changed" failed: exit status 1

This appears to be another case of https://bugs.launchpad.net/charms/+source/ceph-radosgw/+bug/1346489, but for the swift-proxy charm.

An examination of logs shows apt-get update events occurring at the same time as this failure within other units on the same instance.

This was found in a regular CI run while deploying an openstack autopilot cloud: https://ci.lscape.net/job/landscape-system-tests/1508/

Tags: landscape
tags: added: landscape
description: updated
James Page (james-page)
Changed in swift-proxy (Juju Charms Collection):
status: New → Invalid
Revision history for this message
James Page (james-page) wrote :

get_swift_codename does not use any of the retry functionality as done in fetch helper, so is vulnerable to this type of issue.

Changed in charm-swift-proxy:
status: New → Triaged
importance: Undecided → Low
milestone: none → 18.02
Revision history for this message
James Page (james-page) wrote :

Code needs to be moved to using apt_cache helper.

Changed in charm-swift-storage:
status: New → Triaged
importance: Undecided → Low
milestone: none → 18.02
Ryan Beisner (1chb1n)
Changed in charm-swift-proxy:
milestone: 18.02 → 18.05
Changed in charm-swift-storage:
milestone: 18.02 → 18.05
James Page (james-page)
Changed in charm-swift-proxy:
milestone: 18.05 → 18.08
Changed in charm-swift-storage:
milestone: 18.05 → 18.08
James Page (james-page)
Changed in charm-swift-proxy:
milestone: 18.08 → 18.11
Changed in charm-swift-storage:
milestone: 18.08 → 18.11
James Page (james-page)
Changed in charm-swift-proxy:
milestone: 18.11 → 19.04
Changed in charm-swift-storage:
milestone: 18.11 → 19.04
David Ames (thedac)
Changed in charm-swift-proxy:
milestone: 19.04 → 19.07
Changed in charm-swift-storage:
milestone: 19.04 → 19.07
David Ames (thedac)
Changed in charm-swift-proxy:
milestone: 19.07 → 19.10
Changed in charm-swift-storage:
milestone: 19.07 → 19.10
David Ames (thedac)
Changed in charm-swift-proxy:
milestone: 19.10 → 20.01
Changed in charm-swift-storage:
milestone: 19.10 → 20.01
James Page (james-page)
Changed in charm-swift-proxy:
milestone: 20.01 → 20.05
Changed in charm-swift-storage:
milestone: 20.01 → 20.05
David Ames (thedac)
Changed in charm-swift-storage:
milestone: 20.05 → 20.08
David Ames (thedac)
Changed in charm-swift-proxy:
milestone: 20.05 → 20.08
James Page (james-page)
Changed in charm-swift-proxy:
milestone: 20.08 → none
Changed in charm-swift-storage:
milestone: 20.08 → none
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.