Testing on the Queens level of the VMAX driver. Have deployed/created a bunch of servers. When cleaning them up, the terminate_connection for the volume succeeds, but deleting the volume fails. It is going through the same _unlink_vol flow as for bug #1800008, but that bug was opened for a general issue. Here are the details on this case:
- The volume is determined to be a snapvx_target.
- Code loops over the targets (1) in _sync_check()
- break_replication_relationship() is attempted for the full 30 times and each time fails with this message:
Bad or unexpected response from the storage volume backend API: Error modify snapshot resource. The status code received is 500 and the message is {u'message': u'A problem occurred creating the snapshot resource: com.emc.em.common.task.TaskException: The Source device and the Target device do not form a Copy session'}
After timing out like this, the code still attempts to delete the temporary snapshot:
2018-10-29 14:58:46.217 106815 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/dell_emc/vmax/rest.py", line 251, in check_status_code_success
2018-10-29 14:58:46.217 106815 ERROR oslo_messaging.rpc.server data=exception_message)
2018-10-29 14:58:46.217 106815 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error delete snapshot resource. The status code received is 500 and the message is {u'message': u'A problem occurred creating the snapshot resource: com.emc.em.common.task.TaskException: The action cannot be attempted because the snapshot has a link'}.
So these two errors appear to be in conflict - i.e. that there is no Copy session, but yet the snapshot has a link.
After investigating further, it seems that there is not enough information given to delete the snapshot:
self.provision.delete_temp_volume_snap(
array, snap_name, source)
The snap_name is: 'temp-02A9F-volumoot-0'
The source is: '02A9F'
So both input items contain the source volume device. However, this volume was used to clone a lot of volumes, so the temp snapshot name is the same for all of those:
source=[{'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': []}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'030A1']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'03091']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'03085']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'0307B']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'03071']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'03067']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'03058']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'0300F']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02FF6']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02FD5']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02FC2']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02FAC']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02F75']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02F47']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02F10']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02EF8']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02ED2']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02EC8']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02EAC']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02E8C']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02E74']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02E6A']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02E60']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02E54']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02E41']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02D51']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02B4E']}, {'snap_name': u'temp-02A9F-volumoot-0', 'source_vol': u'02A9F', 'target_vol_list': [u'02AC7']}],
So, how can the logic determine which of these snapshots to delete. This is all that is in the request object:
request_obj={'deviceNameListSource': [{'name': u'02A9F'}]}
