New unit fails to see the whole cluster

Bug #2025117 reported by Loïc Gomez
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Repository Cache Charm
Fix Released
High
Haw Loeung

Bug Description

Adding a new unit today, I've been hit by a bug where the new unit failed to join the cluster and see the rest of it.

2023-06-27 05:18:08 INFO unit.ubuntu-repository-cache/6.juju-log server.go:316 cluster:1: Syncing known_hosts @ /home/www-sync/.ssh/known_hosts.
2023-06-27 05:18:08 INFO unit.ubuntu-repository-cache/6.juju-log server.go:316 cluster:1: Updating metadata on a peer
2023-06-27 05:18:08 ERROR unit.ubuntu-repository-cache/6.juju-log server.go:316 cluster:1: Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ubuntu-repository-cache-6/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-ubuntu-repository-cache-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 379, in dispatch
    _invoke(hook_handlers)
  File "/var/lib/juju/agents/unit-ubuntu-repository-cache-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-ubuntu-repository-cache-6/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-ubuntu-repository-cache-6/charm/reactive/ubuntu_repository_cache.py", line 225, in cluster_relation_joined
    mirror.peer_update_metadata()
  File "/var/lib/juju/agents/unit-ubuntu-repository-cache-6/charm/lib/ubuntu_repository_cache/mirror.py", line 371, in peer_update_metadata
    _nonleader_update_metadata()
  File "/var/lib/juju/agents/unit-ubuntu-repository-cache-6/charm/lib/ubuntu_repository_cache/mirror.py", line 276, in _nonleader_update_metadata
    leader_rel = rel[config['leader_unit']]
KeyError: 'ubuntu-repository-cache/4'

Unit is in status hook failed: "cluster-relation-changed"

This is on a unit that properly joined the cluster:
ubuntu@machine-6:~$ sudo juju-run ubuntu-repository-cache/5 "relation-list -r cluster:1"
ubuntu-repository-cache/0
ubuntu-repository-cache/1
ubuntu-repository-cache/2
ubuntu-repository-cache/4
ubuntu-repository-cache/6

This is on the unit failing to join the cluster:
ubuntu@machine-7:~$ sudo juju-run ubuntu-repository-cache/6 "relation-list -r cluster:1"
ubuntu@machine-7:~$

Related branches

Revision history for this message
Loïc Gomez (kotodama) wrote :

Also had to run `juju resolved ubuntu-repository-cache/6` about 8 times or so to get it to resolve and start syncing data.

Haw Loeung (hloeung)
Changed in ubuntu-repository-cache:
status: New → In Progress
assignee: nobody → Haw Loeung (hloeung)
importance: Undecided → High
Haw Loeung (hloeung)
Changed in ubuntu-repository-cache:
status: In Progress → Fix Committed
Haw Loeung (hloeung)
Changed in ubuntu-repository-cache:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.