May 10 10:18:03.304398 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-908b9e12-08da-4f48-bbb2-f3b01c320d92 admin admin] Event 'instance.live_migration_pre.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079176 with metadata {'timestamp': u'2019-05-10 10:18:03.282237', 'message_id': u'90d034b0-d6c0-40b0-9160-e568769c40b5'}
May 10 10:18:18.869791 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-0e809061-4c10-4e82-9c17-20ebf2ca3832 tempest-TestExecuteWorkloadBalancingStrategy-1723816204 tempest-TestExecuteWorkloadBalancingStrategy-1723816204] Event 'instance.delete.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:18.812251', 'message_id': u'999a39cf-602a-404a-82c6-383d0328cb73'}
May 10 10:18:18.869791 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: 1a60bce026aa45ffacf7505275aecb31 {{(pid=2857) __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:283}}
May 10 10:18:18.869791 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-0e809061-4c10-4e82-9c17-20ebf2ca3832 tempest-TestExecuteWorkloadBalancingStrategy-1723816204 tempest-TestExecuteWorkloadBalancingStrategy-1723816204] Event 'instance.shutdown.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:18.819975', 'message_id': u'1cca30c7-dccf-4518-bf35-0ff41d9d9480'}
May 10 10:18:19.753430 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: 28a022d94598481a97195f1ef1eae5e6 {{(pid=2857) __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:283}}
May 10 10:18:19.755615 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-908b9e12-08da-4f48-bbb2-f3b01c320d92 admin admin] Event 'instance.live_migration_post.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:19.768263', 'message_id': u'0dc5bf77-22d9-4b1b-a23e-951c2cd1e2c4'}
May 10 10:18:21.635306 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-0e809061-4c10-4e82-9c17-20ebf2ca3832 tempest-TestExecuteWorkloadBalancingStrategy-1723816204 tempest-TestExecuteWorkloadBalancingStrategy-1723816204] Event 'instance.delete.end' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:21.647710', 'message_id': u'f497122e-df7f-4569-a0e3-33ee4aa249d1'}
In http:// logs.openstack. org/74/ 657374/ 6/check/ watcher- tempest- workload_ balancing/ 3e34c3f/ controller/ logs/screen- watcher- decision- engine. txt.gz
log all the notification events from nova.
It seems devstack deleted the server before it finished the migration.
Maybe we should wait until finished migration before doing other action.
May 10 10:18:03.304398 ubuntu- bionic- vexxhost- sjc1-0006079176 watcher- decision- engine[ 2857]: INFO watcher. decision_ engine. model.notificat ion.nova [None req-908b9e12- 08da-4f48- bbb2-f3b01c320d 92 admin admin] Event 'instance. live_migration_ pre.start' received from nova-compute: ubuntu- bionic- vexxhost- sjc1-0006079176 with metadata {'timestamp': u'2019-05-10 10:18:03.282237', 'message_id': u'90d034b0- d6c0-40b0- 9160-e568769c40 b5'} bionic- vexxhost- sjc1-0006079176 watcher- decision- engine[ 2857]: INFO watcher. decision_ engine. model.notificat ion.nova [None req-0e809061- 4c10-4e82- 9c17-20ebf2ca38 32 tempest- TestExecuteWork loadBalancingSt rategy- 1723816204 tempest- TestExecuteWork loadBalancingSt rategy- 1723816204] Event 'instance. delete. start' received from nova-compute: ubuntu- bionic- vexxhost- sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:18.812251', 'message_id': u'999a39cf- 602a-404a- 82c6-383d0328cb 73'} bionic- vexxhost- sjc1-0006079176 watcher- decision- engine[ 2857]: DEBUG oslo_messaging. _drivers. amqpdriver [-] received message with unique_id: 1a60bce026aa45f facf7505275aecb 31 {{(pid=2857) __call__ /usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ _drivers/ amqpdriver. py:283} } bionic- vexxhost- sjc1-0006079176 watcher- decision- engine[ 2857]: INFO watcher. decision_ engine. model.notificat ion.nova [None req-0e809061- 4c10-4e82- 9c17-20ebf2ca38 32 tempest- TestExecuteWork loadBalancingSt rategy- 1723816204 tempest- TestExecuteWork loadBalancingSt rategy- 1723816204] Event 'instance. shutdown. start' received from nova-compute: ubuntu- bionic- vexxhost- sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:18.819975', 'message_id': u'1cca30c7- dccf-4518- bf35-0ff41d9d94 80'} bionic- vexxhost- sjc1-0006079176 watcher- decision- engine[ 2857]: DEBUG oslo_messaging. _drivers. amqpdriver [-] received message with unique_id: 28a022d94598481 a97195f1ef1eae5 e6 {{(pid=2857) __call__ /usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ _drivers/ amqpdriver. py:283} } bionic- vexxhost- sjc1-0006079176 watcher- decision- engine[ 2857]: INFO watcher. decision_ engine. model.notificat ion.nova [None req-908b9e12- 08da-4f48- bbb2-f3b01c320d 92 admin admin] Event 'instance. live_migration_ post.start' received from nova-compute: ubuntu- bionic- vexxhost- sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:19.768263', 'message_id': u'0dc5bf77- 22d9-4b1b- a23e-951c2cd1e2 c4'} bionic- vexxhost- sjc1-0006079176 watcher- decision- engine[ 2857]: INFO watcher. decision_ engine. model.notificat ion.nova [None req-0e809061- 4c10-4e82- 9c17-20ebf2ca38 32 tempest- TestExecuteWork loadBalancingSt rategy- 1723816204 tempest- TestExecuteWork loadBalancingSt rategy- 1723816204] Event 'instance. delete. end' received from nova-compute: ubuntu- bionic- vexxhost- sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:21.647710', 'message_id': u'f497122e- df7f-4569- a0e3-33ee4aa249 d1'}
May 10 10:18:18.869791 ubuntu-
May 10 10:18:18.869791 ubuntu-
May 10 10:18:18.869791 ubuntu-
May 10 10:18:19.753430 ubuntu-
May 10 10:18:19.755615 ubuntu-
May 10 10:18:21.635306 ubuntu-