It seems that during deployment, keystone didn't respond to ceilometer's request to obtain gnocchi endpoint using gnocchiclient [1] ~~~ 2023-05-08 18:33:49.147 14 WARNING keystoneauth.identity.generic.base [-] Failed to discover available identity versions when contacting http://172.17.1.82:5000. Attempting to parse version from URL.: keystoneauth1.exceptions.connection.ConnectTimeout: Request to http://172.17.1.82:5000 timed out 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base [-] Unable to load publisher gnocchi://?filter_project=service&archive_policy=ceilometer-high: keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. Request to http://172.17.1.82:5000 timed out 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base Traceback (most recent call last): 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base six.raise_from(e, None) 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base File "", line 3, in raise_from 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base httplib_response = conn.getresponse() 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base File "/usr/lib64/python3.9/http/client.py", line 1377, in getresponse 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base response.begin() 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base File "/usr/lib64/python3.9/http/client.py", line 320, in begin 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base version, status, reason = self._read_status() 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base File "/usr/lib64/python3.9/http/client.py", line 281, in _read_status 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base File "/usr/lib64/python3.9/socket.py", line 704, in readinto 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base return self._sock.recv_into(b) 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base socket.timeout: timed out 2023-05-08 18:33:49.150 14 ERROR ceilometer.pipeline.base ~~~ Since ceilometer couldn't get gnocchiclient[2] with proper auth values, it couldn't create the necessary archive policies[3] Restarting agent_notification service after deployment fixes this because by that time keystone is healthy and responding. This seems intermittent because ceilometer & gnocchi services are spawned during step 4 & 5 till then keystone should be completely operational. [1] https://github.com/openstack/ceilometer/blob/stable/wallaby/ceilometer/gnocchi_client.py#L36-L39 [2] https://github.com/openstack/ceilometer/blob/stable/wallaby/ceilometer/publisher/gnocchi.py#L216-L217 [3] https://github.com/openstack/ceilometer/blob/stable/wallaby/ceilometer/publisher/gnocchi.py#L252