This part of the code will never be retried if the Ceph is not bootstrapped at that point.
def _on_relation_changed(self, event): """Enable prometheus on relation change""" if self._charm.unit.is_leader() and ceph_utils.is_bootstrapped(): logger.debug( "is_leader and is_bootstrapped, running rel changed: %s", event ) mgr_config_set_rbd_stats_pools() ceph_utils.mgr_enable_module("prometheus") logger.debug("module_enabled") self.update_alert_rules() super()._on_relation_changed(event)
[when the relation is added at the deployment time (via bundle, etc.)]
# cat /var/log/juju/unit-ceph-mon-0.log | egrep 'metrics-endpoint-relation-|leader-|not bootstrapped'; ll /var/lib/ceph/mon/ceph-ceph-mon-1/done 2024-02-28 10:08:11 INFO juju.worker.uniter.operation runhook.go:186 ran "metrics-endpoint-relation-created" hook (via hook dispatching script: dispatch) 2024-02-28 10:08:16 INFO juju.worker.uniter resolver.go:165 found queued "leader-elected" hook 2024-02-28 10:08:35 INFO juju.worker.uniter.operation runhook.go:186 ran "leader-elected" hook (via hook dispatching script: dispatch) 2024-02-28 10:08:39 INFO unit.ceph-mon/0.juju-log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. 2024-02-28 10:08:40 INFO unit.ceph-mon/0.juju-log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. 2024-02-28 10:08:50 INFO juju.worker.uniter.operation runhook.go:186 ran "metrics-endpoint-relation-joined" hook (via hook dispatching script: dispatch) 2024-02-28 10:09:18 INFO juju.worker.uniter.operation runhook.go:186 ran "metrics-endpoint-relation-changed" hook (via hook dispatching script: dispatch)
-rw-r--r-- 1 root root 0 Feb 28 10:10 /var/lib/ceph/mon/ceph-ceph-mon-1/done
^^^ bootstrap was complete *after* running metrics-endpoint-relation-changed
[when the relation is added after the Ceph model settled]
# cat /var/log/juju/unit-ceph-mon-*.log | egrep 'metrics-endpoint-relation-|leader-|not bootstrapped'; ll /var/lib/ceph/mon/ceph-*/done 2024-02-28 04:43:43 INFO juju.worker.uniter resolver.go:165 found queued "leader-elected" hook 2024-02-28 04:43:46 INFO juju.worker.uniter.operation runhook.go:186 ran "leader-elected" hook (via hook dispatching script: dispatch) 2024-02-28 04:43:48 INFO unit.ceph-mon/1.juju-log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. 2024-02-28 04:43:49 INFO unit.ceph-mon/1.juju-log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. 2024-02-28 04:54:05 INFO juju.worker.uniter.operation runhook.go:186 ran "metrics-endpoint-relation-created" hook (via hook dispatching script: dispatch) 2024-02-28 04:54:07 INFO juju.worker.uniter.operation runhook.go:186 ran "metrics-endpoint-relation-joined" hook (via hook dispatching script: dispatch) 2024-02-28 04:54:15 INFO juju.worker.uniter.operation runhook.go:186 ran "metrics-endpoint-relation-changed" hook (via hook dispatching script: dispatch)
-rw-r--r-- 1 root root 0 Feb 28 04:44 /var/lib/ceph/mon/ceph-juju-704761-1-lxd-1/done
^^^ bootstrap was complete *before* running metrics-endpoint-relation-changed
This part of the code will never be retried if the Ceph is not bootstrapped at that point.
def _on_relation_ changed( self, event): unit.is_ leader( ) and ceph_utils. is_bootstrapped ():
logger. debug(
"is_leader and is_bootstrapped, running rel changed: %s", event
mgr_ config_ set_rbd_ stats_pools( )
ceph_ utils.mgr_ enable_ module( "prometheus" )
logger. debug(" module_ enabled" )
self. update_ alert_rules( )
super( )._on_relation_ changed( event)
"""Enable prometheus on relation change"""
if self._charm.
)
[when the relation is added at the deployment time (via bundle, etc.)]
# cat /var/log/ juju/unit- ceph-mon- 0.log | egrep 'metrics- endpoint- relation- |leader- |not bootstrapped'; ll /var/lib/ ceph/mon/ ceph-ceph- mon-1/done uniter. operation runhook.go:186 ran "metrics- endpoint- relation- created" hook (via hook dispatching script: dispatch) uniter. operation runhook.go:186 ran "leader-elected" hook (via hook dispatching script: dispatch) mon/0.juju- log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. mon/0.juju- log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. uniter. operation runhook.go:186 ran "metrics- endpoint- relation- joined" hook (via hook dispatching script: dispatch) uniter. operation runhook.go:186 ran "metrics- endpoint- relation- changed" hook (via hook dispatching script: dispatch)
2024-02-28 10:08:11 INFO juju.worker.
2024-02-28 10:08:16 INFO juju.worker.uniter resolver.go:165 found queued "leader-elected" hook
2024-02-28 10:08:35 INFO juju.worker.
2024-02-28 10:08:39 INFO unit.ceph-
2024-02-28 10:08:40 INFO unit.ceph-
2024-02-28 10:08:50 INFO juju.worker.
2024-02-28 10:09:18 INFO juju.worker.
-rw-r--r-- 1 root root 0 Feb 28 10:10 /var/lib/ ceph/mon/ ceph-ceph- mon-1/done
^^^ bootstrap was complete *after* running metrics- endpoint- relation- changed
[when the relation is added after the Ceph model settled]
# cat /var/log/ juju/unit- ceph-mon- *.log | egrep 'metrics- endpoint- relation- |leader- |not bootstrapped'; ll /var/lib/ ceph/mon/ ceph-*/ done uniter. operation runhook.go:186 ran "leader-elected" hook (via hook dispatching script: dispatch) mon/1.juju- log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. mon/1.juju- log server.go:325 Ceph is not bootstrapped, skipping upgrade checks. uniter. operation runhook.go:186 ran "metrics- endpoint- relation- created" hook (via hook dispatching script: dispatch) uniter. operation runhook.go:186 ran "metrics- endpoint- relation- joined" hook (via hook dispatching script: dispatch) uniter. operation runhook.go:186 ran "metrics- endpoint- relation- changed" hook (via hook dispatching script: dispatch)
2024-02-28 04:43:43 INFO juju.worker.uniter resolver.go:165 found queued "leader-elected" hook
2024-02-28 04:43:46 INFO juju.worker.
2024-02-28 04:43:48 INFO unit.ceph-
2024-02-28 04:43:49 INFO unit.ceph-
2024-02-28 04:54:05 INFO juju.worker.
2024-02-28 04:54:07 INFO juju.worker.
2024-02-28 04:54:15 INFO juju.worker.
-rw-r--r-- 1 root root 0 Feb 28 04:44 /var/lib/ ceph/mon/ ceph-juju- 704761- 1-lxd-1/ done
^^^ bootstrap was complete *before* running metrics- endpoint- relation- changed