Upgrading from stable to devel charm breaks clients
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
rabbitmq-server (Juju Charms Collection) |
Fix Released
|
Critical
|
James Page |
Bug Description
After upgrading the rabbitmq-server charm from the stable version to the next (devel) version clients could no longer connect.
The rabbit logs showed:
...
=ERROR REPORT==== 3-Aug-2015:
closing AMQP connection <0.13457.0> (10.5.39.129:40991 -> 10.5.39.128:5672):
{handshake_
=ERROR REPORT==== 3-Aug-2015:
closing AMQP connection <0.13461.0> (10.5.39.121:39746 -> 10.5.39.128:5672):
{handshake_
=ERROR REPORT==== 3-Aug-2015:
closing AMQP connection <0.13464.0> (10.5.39.127:43251 -> 10.5.39.128:5672):
{handshake_
...
the password being used by the client was consistent with the password being sent by rabbit:
juju run --unit neutron-gateway/0 "relation-get -r amqp:45 - rabbitmq-server/0"
hostname: 10.5.39.128
password: YxMpxJtTc33g637
private-address: 10.5.39.128
juju run --unit neutron-gateway/0 "grep rabbit /etc/neutron/
rabbit_userid = neutron
rabbit_virtual_host = openstack
rabbit_password = YxMpxJtTc33g637
rabbit_host = 10.5.39.128
list_notifier_
changing the password via rabbitmqctl to the one being used by the client fixed the issue
Related branches
- Edward Hope-Morley: Needs Fixing
- Liam Young (community): Approve
-
Diff: 98 lines (+12/-11)2 files modifiedcharmhelpers/contrib/peerstorage/__init__.py (+5/-4)
tests/contrib/peerstorage/test_peerstorage.py (+7/-7)
- charmers: Pending requested
-
Diff: 96 lines (+37/-6)3 files modifiedhooks/charmhelpers/contrib/peerstorage/__init__.py (+5/-4)
hooks/charmhelpers/core/hookenv.py (+16/-1)
tests/charmhelpers/core/hookenv.py (+16/-1)
- charmers: Pending requested
-
Diff: 4321 lines (+2410/-456)36 files modifiedcharm-helpers.yaml (+1/-0)
hooks/charmhelpers/cli/__init__.py (+195/-0)
hooks/charmhelpers/cli/benchmark.py (+36/-0)
hooks/charmhelpers/cli/commands.py (+32/-0)
hooks/charmhelpers/cli/host.py (+31/-0)
hooks/charmhelpers/cli/unitdata.py (+39/-0)
hooks/charmhelpers/contrib/hahelpers/cluster.py (+47/-3)
hooks/charmhelpers/contrib/openstack/amulet/deployment.py (+42/-5)
hooks/charmhelpers/contrib/openstack/amulet/utils.py (+361/-51)
hooks/charmhelpers/contrib/openstack/context.py (+47/-34)
hooks/charmhelpers/contrib/openstack/ip.py (+49/-44)
hooks/charmhelpers/contrib/openstack/neutron.py (+16/-9)
hooks/charmhelpers/contrib/openstack/templates/ceph.conf (+6/-6)
hooks/charmhelpers/contrib/openstack/templating.py (+2/-2)
hooks/charmhelpers/contrib/openstack/utils.py (+88/-24)
hooks/charmhelpers/contrib/peerstorage/__init__.py (+124/-3)
hooks/charmhelpers/contrib/python/packages.py (+30/-5)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+6/-6)
hooks/charmhelpers/contrib/storage/linux/utils.py (+1/-1)
hooks/charmhelpers/core/files.py (+45/-0)
hooks/charmhelpers/core/hookenv.py (+352/-44)
hooks/charmhelpers/core/host.py (+56/-12)
hooks/charmhelpers/core/services/base.py (+43/-19)
hooks/charmhelpers/core/services/helpers.py (+2/-2)
hooks/charmhelpers/core/unitdata.py (+61/-17)
hooks/charmhelpers/fetch/__init__.py (+24/-15)
hooks/charmhelpers/fetch/archiveurl.py (+7/-1)
hooks/charmhelpers/fetch/giturl.py (+8/-6)
hooks/rabbit_utils.py (+24/-14)
hooks/rabbitmq_server_relations.py (+76/-39)
tests/charmhelpers/core/files.py (+45/-0)
tests/charmhelpers/core/hookenv.py (+352/-44)
tests/charmhelpers/core/host.py (+56/-12)
tests/charmhelpers/core/services/base.py (+43/-19)
tests/charmhelpers/core/services/helpers.py (+2/-2)
tests/charmhelpers/core/unitdata.py (+61/-17)
Changed in rabbitmq-server (Juju Charms Collection): | |
importance: | Undecided → Critical |
milestone: | none → 15.07 |
status: | New → Confirmed |
Changed in rabbitmq-server (Juju Charms Collection): | |
status: | Confirmed → In Progress |
assignee: | nobody → James Page (james-page) |
Changed in rabbitmq-server (Juju Charms Collection): | |
status: | In Progress → Fix Committed |
Changed in rabbitmq-server (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Cluster and leader storage is inconsistent:
ubuntu@ james-page- bastion: ~/charms⟫ juju run --service rabbitmq-server "relation-get -r cluster:12 - rabbitmq-server/0" P4nk72n4jr838Nk W6N5R88R3mBHzK7 LYG42849f3cPGkz 6Ztg nVX38FPCPfbHGZc C69KNmkPn2z5Vg3 7n3f5dfYZdfnXTV b4yV JywJWqfpd5wVt6X wpV8LY88mqjJLys qbR8ML9mGq3Wnxw 3X3r rabbitmq- server- 0.passwd: fgMPL6bqRYJjYFL cKtmYm433zW9qfb V93SWYngytF8sHR XY9FTjJg2HBybFK ZFpz Mfk7JTpc3CKm3dg TgMhndCjNZnmHnN xfHHkkhfSmxFpfj nTBd V8bCMF39LBTh7cb kjHy4nKmtJ9rWNy V5xdLx8F2ZxHRWH hMy8
ceilometer.passwd: SWRwZPpSr3sbZtB
cinder.passwd: YhXCqgYz9MtMgKq
heat.passwd: dZFf2zWHMpWHrK2
nagios-
neutron.passwd: 6fp58xNC54B5y46
nova.passwd: 3fb57LtWMxYHkk6
private-address: 10.5.26.108
ubuntu@ james-page- bastion: ~/charms⟫ juju run --service rabbitmq-server "leader-get" get_migrated_ settings_ _: '["nova.passwd", "neutron.passwd"]' NSJRC3ZBk6VYgzC nNd8NNr62XZCcSV NHbJwk9G6tVrhgj 4qJz rkhd9Jy2TcRM8jC cwG2GPnz97SC5rd 2KWC5ZK6CTn8ckL KZYY Tt5FpgyWbSkrksf dTTmqHXdnRzVrKw CNBz8GZ9K84gHwL CtM8 NJd73yNRnpjty89 VZTmwZkz3Yscy5p n8nH25qJ9Nt4g9J NRCG sHR96Ss9XymFypL SprZctJk6TbKck2 rf2Zh9xPggzYGGZ qYbR
__leader_
ceilometer.passwd: hscCZxjpV42W4B8
cinder.passwd: kKrpY27LjmV5PXL
heat.passwd: B3r2xkft5pVfSPY
neutron.passwd: 43ykPZYhjyhpkjS
nova.passwd: tMRPBgXXzn2rwbS