Cinder

[CEPH] cinder-backup service blocked by creating volume from backup

Bug #1939842 reported by Jeremy Liu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
In Progress
Low
Jeremy Liu

Bug Description

When creating a large volume(1000GB) from a backup, cinder-backup service no longer send heartbeat to cinder-scheduler in a perioid of time. Further debug shows the backup restore process hangs at librbd diff_iterate function when it tries to calcuate whether the given rbd volume has extents.

volume driver: rbd
backup driver: ceph

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/804470

Changed in cinder:
status: New → In Progress
Sofia Enriquez (lsofia-enriquez)
Changed in cinder:
importance: Undecided → Low
Sofia Enriquez (lsofia-enriquez)
summary: - cinder-backup service blocked by creating volume from backup
+ [CEPH] cinder-backup service blocked by creating volume from backup
tags: added: backup-service ceph rbd restore
Revision history for this message
Sofia Enriquez (lsofia-enriquez) wrote :

Greetings Jeremy Liu,
Do you mind sharing with us more detailed information. In this way we could reproduce or help with this.
- steps to reproduce the problem.
- logs: c-api c-bak and c-vol and any other information you think It may help us.- cinder conf
- from the patch proposed in master I guess you're facing this issue on master but I need to ask: which cinder version are you referring to?
Thanks in advance,Sofia

Changed in cinder:
status: In Progress → Incomplete
assignee: nobody → Jeremy Liu (liujiong)
Revision history for this message
Jeremy Liu (liujiong) wrote :

I'm testing against cinder rocky release, but I suppose this also affect master.
volume driver: rbd
backup driver: ceph

steps to reproduce:
1. create a large volume, cinder create --name large-volume --availability-zone az01 1000
2. attach the volume to instance, mount volume to instance file system, write data into volume, make volume usage to about 50%
3. create a backup for the volume
4. create volume from the backup, cinder create --backup-id <backup-uuid> --name volume-from-backup 1000

Then watch cinder service-list command output.

OpenStack Infra (hudson-openstack)
Changed in cinder:
status: Incomplete → In Progress
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.