Comment 3 for bug 2016138

Revision history for this message
Christian Rohmann (christian-rohmann) wrote :

We just had a list of backups left in the "creating" state even though cinder-backup was not working on them anymore (is there not check-loop to see if the individual backup process is still alive?).

I restarted cinder-backup to have it move all those backups to the "error" state on startup.
But some volumes were already deleted in the meantime, causing SQL error:

2023-06-21 08:24:35.353 1345602 INFO cinder.backup.manager [req-cfd983eb-a261-495f-bada-ad52ac04f866 - - - - -] Resetting backup fe1b42f6-49ee-4f24-9220-841c37c25928 to error (was creating).
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager [req-cfd983eb-a261-495f-bada-ad52ac04f866 - - - - -] Problem cleaning up backup fe1b42f6-49ee-4f24-9220-841c37c25928.: cinder.exception.VolumeNotFound: Volume f7b070c4-daea-4f0b-a8d9-95470f750e83 could not be found.
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager Traceback (most recent call last):
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 202, in _cleanup_incomplete_backup_operations
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager self._cleanup_one_backup(ctxt, backup)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 235, in _cleanup_one_backup
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager volume = objects.Volume.get_by_id(ctxt, backup.volume_id)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/objects/base.py", line 339, in get_by_id
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager orm_obj = db.get_by_id(context, cls.model, id, *args, **kwargs)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/db/api.py", line 109, in get_by_id
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager return IMPL.get_by_id(context, model, id, *args, **kwargs)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 187, in wrapper
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager return f(*args, **kwargs)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 427, in get_by_id
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager return _GET_METHODS[model](context, id, *args, **kwargs)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 187, in wrapper
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager return f(*args, **kwargs)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 2367, in volume_get
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager return _volume_get(context, volume_id)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 187, in wrapper
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager return f(*args, **kwargs)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 2126, in _volume_get
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager raise exception.VolumeNotFound(volume_id=volume_id)
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager cinder.exception.VolumeNotFound: Volume f7b070c4-daea-4f0b-a8d9-95470f750e83 could not be found.
2023-06-21 08:24:35.359 1345602 ERROR cinder.backup.manager