I upgraded nova-compute nodes to test live migration patch
- HTTPS_PROXY=http://172.31.254.9:8080/ sudo -E apt-add-repository ppa:billy-olsen/lp1888395-train
- sudo apt install nova-api-metadata nova-common nova-compute nova-compute-kvm nova-compute-libvirt python3-nova
- sudo systemctl restart nova-*
Created VM and tried to live-migrate it
- openstack server create --image auto-sync/ubuntu-xenial-16.04-amd64-server-20200922-disk1.img --network VN_Red --flavor g1t1.small --boot-from-volume 3 --availability-zone zone1 vern1
- openstack server migrate vern1 --live-migration
- nova.exception.InternalError: Failure running os_vif plugin unplug method: Failed to unplug VIF VIFGeneric(active=True,address=02:5b:d4:23:c6:b1,has_traffic_filtering=True,id=5bd423c6-b16d-4678-a05a-3ab94af82d4a,network=Network(e593a2d7-f9e0-4038-957a-1378977bc314),plugin='vrouter',port_profile=<?>,preserve_on_delete=False,vif_name='tap5bd423c6-b1'). Got error: Error during the call to vrouter-port-control: ('vrouter-port-control', '--oper=delete', '--uuid=5bd423c6-b16d-4678-a05a-3ab94af82d4a')
Upgraded nova-cloud-controller units as well
- HTTPS_PROXY=http://172.31.254.9:8080/ sudo -E apt-add-repository ppa:billy-olsen/lp1888395-train
- sudo apt install nova-api-os-compute nova-common nova-conductor nova-scheduler nova-spiceproxy python3-nova
- sudo systemctl restart nova-* apache2
Retrying the above create and migrate commands
- Same error.
Full trace from `openstack server show vern1 -f value -c fault | sed 's/\\n/\n/g;s/\\'\''/'\''/g'`:
{'code': 500, 'created': '2020-10-27T06:16:17Z', 'message': "Failure running os_vif plugin unplug method: Failed to unplug VIF VIFGeneric(active=True,address=02:26:27:d1:5a:36,has_traffic_filtering=True,id=2627d15a-3604-4a11-80d7-12b92ed9cea6,network=Network(e593a2d7-f9e0-4038-957a-1378977bc314),plugin='vrouter',po", 'details': 'Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 110, in unplug
plugin.unplug(vif, instance_info)
File "/usr/lib/python3/dist-packages/vif_plug_vrouter/vrouter.py", line 301, in unplug
self._vrouter_port_delete(instance_info, vif)
File "/usr/lib/python3/dist-packages/vif_plug_vrouter/vrouter.py", line 293, in _vrouter_port_delete
vhostuser_socket, vhostuser_mode)
File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 245, in _wrap
return self.channel.remote_call(name, args, kwargs)
File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 204, in remote_call
raise exc_type(*result[2])
vif_plug_vrouter.exception.VrouterPortControlError: Error during the call to vrouter-port-control: ('vrouter-port-control', '--oper=delete', '--uuid=2627d15a-3604-4a11-80d7-12b92ed9cea6')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 820, in _unplug_os_vif
os_vif.unplug(vif, instance_info)
File "/usr/lib/python3/dist-packages/os_vif/__init__.py", line 115, in unplug
raise os_vif.exception.UnplugException(vif=vif, err=err)
os_vif.exception.UnplugException: Failed to unplug VIF VIFGeneric(active=True,address=02:26:27:d1:5a:36,has_traffic_filtering=True,id=2627d15a-3604-4a11-80d7-12b92ed9cea6,network=Network(e593a2d7-f9e0-4038-957a-1378977bc314),plugin='vrouter',port_profile=<?>,preserve_on_delete=False,vif_name='tap2627d15a-36'). Got error: Error during the call to vrouter-port-control: ('vrouter-port-control', '--oper=delete', '--uuid=2627d15a-3604-4a11-80d7-12b92ed9cea6')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 207, in decorated_function
return function(self, context, *args, **kwargs)
File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 7546, in _post_live_migration
unplug_nw_info)
File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 9489, in post_live_migration_at_source
self.unplug_vifs(instance, network_info)
File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 1152, in unplug_vifs
self._unplug_vifs(instance, network_info, False)
File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 1146, in _unplug_vifs
self.vif_driver.unplug(instance, vif)
File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 844, in unplug
self._unplug_os_vif(instance, vif_obj)
File "/usr/lib/python3/dist-packages/nova/virt/libvirt/vif.py", line 824, in _unplug_os_vif
raise exception.InternalError(msg)
nova.exception.InternalError: Failure running os_vif plugin unplug method: Failed to unplug VIF VIFGeneric(active=True,address=02:26:27:d1:5a:36,has_traffic_filtering=True,id=2627d15a-3604-4a11-80d7-12b92ed9cea6,network=Network(e593a2d7-f9e0-4038-957a-1378977bc314),plugin='vrouter',port_profile=<?>,preserve_on_delete=False,vif_name='tap2627d15a-36'). Got error: Error during the call to vrouter-port-control: ('vrouter-port-control', '--oper=delete', '--uuid=2627d15a-3604-4a11-80d7-12b92ed9cea6')
'}
I upgraded nova-compute nodes to test live migration patch 172.31. 254.9:8080/ sudo -E apt-add-repository ppa:billy- olsen/lp1888395 -train libvirt python3-nova
- HTTPS_PROXY=http://
- sudo apt install nova-api-metadata nova-common nova-compute nova-compute-kvm nova-compute-
- sudo systemctl restart nova-*
Created VM and tried to live-migrate it ubuntu- xenial- 16.04-amd64- server- 20200922- disk1.img --network VN_Red --flavor g1t1.small --boot-from-volume 3 --availability-zone zone1 vern1 InternalError: Failure running os_vif plugin unplug method: Failed to unplug VIF VIFGeneric( active= True,address= 02:5b:d4: 23:c6:b1, has_traffic_ filtering= True,id= 5bd423c6- b16d-4678- a05a-3ab94af82d 4a,network= Network( e593a2d7- f9e0-4038- 957a-1378977bc3 14),plugin= 'vrouter' ,port_profile= <?>,preserve_ on_delete= False,vif_ name='tap5bd423 c6-b1') . Got error: Error during the call to vrouter- port-control: ('vrouter- port-control' , '--oper=delete', '--uuid= 5bd423c6- b16d-4678- a05a-3ab94af82d 4a')
- openstack server create --image auto-sync/
- openstack server migrate vern1 --live-migration
- nova.exception.
Upgraded nova-cloud- controller units as well 172.31. 254.9:8080/ sudo -E apt-add-repository ppa:billy- olsen/lp1888395 -train
- HTTPS_PROXY=http://
- sudo apt install nova-api-os-compute nova-common nova-conductor nova-scheduler nova-spiceproxy python3-nova
- sudo systemctl restart nova-* apache2
Retrying the above create and migrate commands
- Same error.
Full trace from `openstack server show vern1 -f value -c fault | sed 's/\\n/ \n/g;s/ \\'\''/ '\''/g' `:
{'code': 500, 'created': '2020-10- 27T06:16: 17Z', 'message': "Failure running os_vif plugin unplug method: Failed to unplug VIF VIFGeneric( active= True,address= 02:26:27: d1:5a:36, has_traffic_ filtering= True,id= 2627d15a- 3604-4a11- 80d7-12b92ed9ce a6,network= Network( e593a2d7- f9e0-4038- 957a-1378977bc3 14),plugin= 'vrouter' ,po", 'details': 'Traceback (most recent call last): python3/ dist-packages/ os_vif/ __init_ _.py", line 110, in unplug unplug( vif, instance_info) python3/ dist-packages/ vif_plug_ vrouter/ vrouter. py", line 301, in unplug _vrouter_ port_delete( instance_ info, vif) python3/ dist-packages/ vif_plug_ vrouter/ vrouter. py", line 293, in _vrouter_ port_delete socket, vhostuser_mode) python3/ dist-packages/ oslo_privsep/ priv_context. py", line 245, in _wrap remote_ call(name, args, kwargs) python3/ dist-packages/ oslo_privsep/ daemon. py", line 204, in remote_call *result[ 2]) vrouter. exception. VrouterPortCont rolError: Error during the call to vrouter- port-control: ('vrouter- port-control' , '--oper=delete', '--uuid= 2627d15a- 3604-4a11- 80d7-12b92ed9ce a6')
File "/usr/lib/
plugin.
File "/usr/lib/
self.
File "/usr/lib/
vhostuser_
File "/usr/lib/
return self.channel.
File "/usr/lib/
raise exc_type(
vif_plug_
During handling of the above exception, another exception occurred:
Traceback (most recent call last): python3/ dist-packages/ nova/virt/ libvirt/ vif.py" , line 820, in _unplug_os_vif vif.unplug( vif, instance_info) python3/ dist-packages/ os_vif/ __init_ _.py", line 115, in unplug exception. UnplugException (vif=vif, err=err) exception. UnplugException : Failed to unplug VIF VIFGeneric( active= True,address= 02:26:27: d1:5a:36, has_traffic_ filtering= True,id= 2627d15a- 3604-4a11- 80d7-12b92ed9ce a6,network= Network( e593a2d7- f9e0-4038- 957a-1378977bc3 14),plugin= 'vrouter' ,port_profile= <?>,preserve_ on_delete= False,vif_ name='tap2627d1 5a-36') . Got error: Error during the call to vrouter- port-control: ('vrouter- port-control' , '--oper=delete', '--uuid= 2627d15a- 3604-4a11- 80d7-12b92ed9ce a6')
File "/usr/lib/
os_
File "/usr/lib/
raise os_vif.
os_vif.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): python3/ dist-packages/ nova/compute/ manager. py", line 207, in decorated_function python3/ dist-packages/ nova/compute/ manager. py", line 7546, in _post_live_ migration python3/ dist-packages/ nova/virt/ libvirt/ driver. py", line 9489, in post_live_ migration_ at_source unplug_ vifs(instance, network_info) python3/ dist-packages/ nova/virt/ libvirt/ driver. py", line 1152, in unplug_vifs _unplug_ vifs(instance, network_info, False) python3/ dist-packages/ nova/virt/ libvirt/ driver. py", line 1146, in _unplug_vifs vif_driver. unplug( instance, vif) python3/ dist-packages/ nova/virt/ libvirt/ vif.py" , line 844, in unplug _unplug_ os_vif( instance, vif_obj) python3/ dist-packages/ nova/virt/ libvirt/ vif.py" , line 824, in _unplug_os_vif InternalError( msg) InternalError: Failure running os_vif plugin unplug method: Failed to unplug VIF VIFGeneric( active= True,address= 02:26:27: d1:5a:36, has_traffic_ filtering= True,id= 2627d15a- 3604-4a11- 80d7-12b92ed9ce a6,network= Network( e593a2d7- f9e0-4038- 957a-1378977bc3 14),plugin= 'vrouter' ,port_profile= <?>,preserve_ on_delete= False,vif_ name='tap2627d1 5a-36') . Got error: Error during the call to vrouter- port-control: ('vrouter- port-control' , '--oper=delete', '--uuid= 2627d15a- 3604-4a11- 80d7-12b92ed9ce a6')
File "/usr/lib/
return function(self, context, *args, **kwargs)
File "/usr/lib/
unplug_nw_info)
File "/usr/lib/
self.
File "/usr/lib/
self.
File "/usr/lib/
self.
File "/usr/lib/
self.
File "/usr/lib/
raise exception.
nova.exception.
'}