Unfortunately we couldn't reproduce the issue in our lab. We saw this on customer's env (Fuel 9.0).
Happens randomly when the array is handling multiple requests to create and attach volumes at the same time.
2016-09-07 14:18:59.661 22217 INFO cinder.volume.manager [req-5016e25a-4c33-4670-8844-65fc1a6f19dc 1f0a08659d2a4ce1a3523c2637f572be 3b93a86a24e84cd4afae7d336d42fc31 - - -] Terminate volume connection completed successfully. 2016-09-07 14:18:59.668 22218 ERROR cinder.volume.manager [req-14d00815-2c96-48a9-a119-d5e2da8646b2 1f0a08659d2a4ce1a3523c2637f572be 3b93a86a24e84cd4afae7d336d42fc31 - - -] Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_admin_metadata' cannot proceed 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher [req-14d00815-2c96-48a9-a119-d5e2da8646b2 1f0a08659d2a4ce1a3523c2637f572be 3b93a86a24e84cd4afae7d336d42fc31 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_admin_metadata' cannot proceed 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1541, in terminate_connection 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher raise exception.VolumeBackendAPIException(data=err_msg) 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_admin_metadata' cannot proceed 2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.rpc.dispatcher 2016-09-07 14:18:59.673 22218 ERROR oslo_messaging._drivers.common [req-14d00815-2c96-48a9-a119-d5e2da8646b2 1f0a08659d2a4ce1a3523c2637f572be 3b93a86a24e84cd4afae7d336d42fc31 - - -] Returning exception Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_admin_metadata' cannot proceed to caller 2016-09-07 14:18:59.674 22218 ERROR oslo_messaging._drivers.common [req-14d00815-2c96-48a9-a119-d5e2da8646b2 1f0a08659d2a4ce1a3523c2637f572be 3b93a86a24e84cd4afae7d336d42fc31 - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1541, in terminate_connection\n raise exception.VolumeBackendAPIException(data=err_msg)\n', "VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_admin_metadata' cannot proceed\n"] 2016-09-07 14:18:59.762 22218 INFO cinder.volume.drivers.huawei.rest_client [req-e271a795-f198-4387-8662-729968818dd9 1f0a08659d2a4ce1a3523c2637f572be 3b93a86a24e84cd4afae7d336d42fc31 - - -]
Unfortunately we couldn't reproduce the issue in our lab. We saw this on customer's env (Fuel 9.0).
Happens randomly when the array is handling multiple requests to create and attach volumes at the same time.
2016-09-07 14:18:59.661 22217 INFO cinder. volume. manager [req-5016e25a- 4c33-4670- 8844-65fc1a6f19 dc 1f0a08659d2a4ce 1a3523c2637f572 be 3b93a86a24e84cd 4afae7d336d42fc 31 - - -] Terminate volume connection completed successfully. volume. manager [req-14d00815- 2c96-48a9- a119-d5e2da8646 b2 1f0a08659d2a4ce 1a3523c2637f572 be 3b93a86a24e84cd 4afae7d336d42fc 31 - - -] Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_ admin_metadata' cannot proceed rpc.dispatcher [req-14d00815- 2c96-48a9- a119-d5e2da8646 b2 1f0a08659d2a4ce 1a3523c2637f572 be 3b93a86a24e84cd 4afae7d336d42fc 31 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_ admin_metadata' cannot proceed rpc.dispatcher Traceback (most recent call last): rpc.dispatcher File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 138, in _dispatch_and_reply rpc.dispatcher incoming.message)) rpc.dispatcher File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 185, in _dispatch rpc.dispatcher return self._do_ dispatch( endpoint, method, ctxt, args) rpc.dispatcher File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 127, in _do_dispatch rpc.dispatcher result = func(ctxt, **new_args) rpc.dispatcher File "/usr/lib/ python2. 7/dist- packages/ cinder/ volume/ manager. py", line 1541, in terminate_ connection rpc.dispatcher raise exception. VolumeBackendAP IException( data=err_ msg) rpc.dispatcher VolumeBackendAP IException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_ admin_metadata' cannot proceed rpc.dispatcher _drivers. common [req-14d00815- 2c96-48a9- a119-d5e2da8646 b2 1f0a08659d2a4ce 1a3523c2637f572 be 3b93a86a24e84cd 4afae7d336d42fc 31 - - -] Returning exception Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_ admin_metadata' cannot proceed to caller _drivers. common [req-14d00815- 2c96-48a9- a119-d5e2da8646 b2 1f0a08659d2a4ce 1a3523c2637f572 be 3b93a86a24e84cd 4afae7d336d42fc 31 - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 138, in _dispatch_ and_reply\ n incoming. message) )\n', ' File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 185, in _dispatch\n return self._do_ dispatch( endpoint, method, ctxt, args)\n', ' File "/usr/lib/ python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/ python2. 7/dist- packages/ cinder/ volume/ manager. py", line 1541, in terminate_ connection\ n raise exception. VolumeBackendAP IException( data=err_ msg)\n' , "VolumeBackendA PIException: Bad or unexpected response from the storage volume backend API: Terminate volume connection failed: Parent instance <Volume at 0x7fad3b59ec10> is not bound to a Session; lazy load operation of attribute 'volume_ admin_metadata' cannot proceed\n"] volume. drivers. huawei. rest_client [req-e271a795- f198-4387- 8662-729968818d d9 1f0a08659d2a4ce 1a3523c2637f572 be 3b93a86a24e84cd 4afae7d336d42fc 31 - - -]
2016-09-07 14:18:59.668 22218 ERROR cinder.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.669 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.673 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.674 22218 ERROR oslo_messaging.
2016-09-07 14:18:59.762 22218 INFO cinder.