upgrade-juju fails on all providers

Bug #1309108 reported by Aaron Bentley
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
juju-core
Fix Released
Critical
Andrew Wilkins

Bug Description

upgrade-juju is not upgrading the agents in a reasonable time. It may upgrade some, but not all of them. In this build, it upgraded machine 2's agent, but not machines 0, or 1, or any unit agents.
http://ec2-54-84-137-170.compute-1.amazonaws.com:8080/job/local-upgrade/1103/console

This has been happening with our builds of r2644 and later. r2642 and earlier appear unaffected.

Revision history for this message
Aaron Bentley (abentley) wrote :
Curtis Hovey (sinzui)
Changed in juju-core:
milestone: none → 1.19.1
summary: - upgrade-juju fails on local provider
+ upgrade-juju fails on all providers
Revision history for this message
Curtis Hovey (sinzui) wrote :
Revision history for this message
Curtis Hovey (sinzui) wrote :
Revision history for this message
Curtis Hovey (sinzui) wrote :
Revision history for this message
Curtis Hovey (sinzui) wrote :
Revision history for this message
Curtis Hovey (sinzui) wrote :
Revision history for this message
John A Meinel (jameinel) wrote :

I'm wondering if your script is detecting the right thing, looking at this console output:
http://ec2-54-84-137-170.compute-1.amazonaws.com:8080/job/hp-upgrade/1089/console

During the last upgrade it gets to:
juju --show-log upgrade-juju -e test-release-hp --version 1.19.1
...
+ PATH=/mnt/jenkinshome/jobs/hp-upgrade/workspace/extracted-bin/usr/lib/juju-1.19.1/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
+ /var/lib/jenkins/ci-cd-scripts2/wait_for_agent_update.py test-release-hp
1.18.1: 1, 0, 2, dummy-sink/0, dummy-source/0
1.18.1: 1, 0, 2, dummy-sink/0, dummy-source/0
1.18.1: 1, 0, 2, dummy-sink/0, dummy-source/0
1.18.1: 1, 0, 2, dummy-sink/0, dummy-source/0
1.18.1: 1, 0, 2, dummy-sink/0, dummy-source/0
1.18.1: 1, 0, 2, dummy-sink/0, dummy-source/0
1.18.1: 0, dummy-sink/0, dummy-source/0
1.18.1: 0, dummy-sink/0, dummy-source/0
1.18.1: 0, dummy-sink/0, dummy-source/0
1.18.1: 0, dummy-sink/0, dummy-source/0
1.18.1: 0, dummy-sink/0, dummy-source/0
1.18.1: 0, dummy-sink/0, dummy-source/0
1.18.1: 0
1.18.1: 0
1.18.1: 0
1.18.1: 0

I was misunderstanding the output. It isn't that 0 items haven't upgrade, but instead that the agent calling itself '0' (aka machine agent for machine-0) is still on 1.18.1. That is really strange. It does seem to fit what I'm seeing in the log files, though. What is weird is that I see several machines upgrade (they at least stop being 1.18.1, perhaps they are just dead. It might be useful to have slightly different output here.)

I haven't been able to reproduce this (upgrade succeeds in the various tests that I've tried locally and on AWS), but I'll keep poking at it.

Revision history for this message
Curtis Hovey (sinzui) wrote :

lp:juju-core r2655 fixed upgrade. I don't see a reason why this rev fixed the issue. Nate and I thought lp:juju-core r2654 was the more likely reason, but test results show that all upgrades failed with the earlier rev.

Changed in juju-core:
status: Triaged → Fix Released
John A Meinel (jameinel)
Changed in juju-core:
assignee: nobody → Andrew Wilkins (axwalk)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.