Mirantis OpenStack

[Cinder] users that has _member_ privileges cannot extend Volumes randomly

Bug #1597742 reported by Arvydas
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Invalid
High
MOS Cinder
Mirantis OpenStack 9.2
8.0.x
Invalid
High
MOS Cinder
Mirantis OpenStack 8.0-updates
9.x
Invalid
High
MOS Cinder
Mirantis OpenStack 9.2

Bug Description

Detailed bug description:

If user is added to the project as _member_ then Volume extension fails very often. Only "admin" can extend without errors

Steps to reproduce:

1. Create project and Create user for this project.
2. Add created user with _member_ role
3. Create volume:

~# cinder create --name TestHDD 10

~# cinder list|grep TestHDD
| 00de505d-f76e-4a69-bcd6-3bd1dac0d340 | available | TestHDD | 10 |

4. Extend volume:

~# cinder extend TestHDD 20

5. See the result:

~# cinder list|grep TestHDD
| 00de505d-f76e-4a69-bcd6-3bd1dac0d340 | error_extending | TestHDD | 10 |

Expected results:

Volume extension is successful.

Actual result:

Volume extension fails (not every time, but every second or third time)

cinder-volume.log contents:

2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager [req-40848326-9d95-4901-a9c3-49b444ae975b 362232471834441588ee57c24bd63deb fe1ad187815b4f44b5d1c1affa2cc167 - - -] Extend volume failed.
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager Traceback (most recent call last):
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1992, in extend_volume
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager return f(*args, **kwargs)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/huawei/huawei_driver.py", line 796, in extend_volume
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager volume, constants.VOLUME_NOT_EXISTS_RAISE)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/huawei/huawei_driver.py", line 778, in _check_volume_exist_on_array
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager metadata = huawei_utils.get_admin_metadata(volume)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/huawei/huawei_utils.py", line 93, in get_admin_metadata
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager elif 'volume_admin_metadata' in volume:
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/models.py", line 61, in __contains__
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager getattr(self, key)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 237, in __get__
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager return self.impl.get(instance_state(instance), dict_)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 578, in get
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager value = self.callable_(state, passive)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 502, in _load_for_state
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager (orm_util.state_str(state), self.key)
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager DetachedInstanceError: Parent instance <Volume at 0x7fc79dedc9d0> is not bound to a Session; lazy load operation of attribute 'volume_admin_metadata' cannot proceed
2016-06-30 10:45:03.012 15094 ERROR cinder.volume.manager

Reproducibility:

98% (sometimes IT GETS EXTENDED)

Impact:

Member of the Porject cannot extend Volumes

Description of the environment:

MOS 8
- OS: UBUNTU 14.04
open-iscsi 2.0.873-3ubuntu9
cinder-api 2:7.0.1-1~u14.04+mos38
cinder-backup 2:7.0.1-1~u14.04+mos38
cinder-common 2:7.0.1-1~u14.04+mos38
cinder-scheduler 2:7.0.1-1~u14.04+mos38
cinder-volume 2:7.0.1-1~u14.04+mos38
python-cinder 2:7.0.1-1~u14.04+mos38
python-cinderclient 1:1.4.0-1~u14.04+mos3
multipath-tools 0.4.9-3ubuntu7.12
volume_driver cinder.volume.drivers.huawei.huawei_driver.HuaweiISCSIDriver
keystone 2:8.1.0-2~u14.04+mos4
python-keystone 2:8.1.0-2~u14.04+mos4
python-keystoneclient 1:1.7.4-2~u14.04+mos0
python-keystonemiddleware 2.3.3-2~u14.04+mos0

Additional information:

if added user is with "admin" role - no extending error occurs.

Dina Belova (dbelova)
tags: added: area-cinder
Revision history for this message
Oleksiy Butenko (obutenko) wrote :

can you upload diag-snapshot?

tags: added: need-info
Revision history for this message
Yuriy Nesenenko (ynesenenko) wrote :

Please add logs.

Revision history for this message
Arvydas (arvydas-brazenas-8) wrote :

I can't upload diag-snapshot, since i get:

[root@fuel ~]# fuel snapshot --conf > dump_conf.yaml
[root@fuel ~]# shotgun -c dump_conf.yaml
Traceback (most recent call last):
  File "/usr/bin/shotgun", line 9, in <module>
    load_entry_point('shotgun==8.0.0', 'console_scripts', 'shotgun')()
  File "/usr/lib/python2.7/site-packages/shotgun/cli.py", line 71, in main
    make_snapshot(parse_args())
  File "/usr/lib/python2.7/site-packages/shotgun/cli.py", line 63, in make_snapshot
    config_object = Config(read_config(args.config))
  File "/usr/lib/python2.7/site-packages/shotgun/cli.py", line 53, in read_config
    config = json.loads(fo.read())
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Attaching cinder logs

Denis Meltsaykin (dmeltsaykin)
tags: removed: need-info
Revision history for this message
Ivan Kolodyazhny (e0ne) wrote :

Arvydas, could you please add more details to this issue? Unfortunately, we don't have environment with Huawei storage to reproduce this issue

Ivan Kolodyazhny (e0ne)
tags: added: move-to-9.2
Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Marking as 'Invalid' because there is not enough information for 4 months. Please move back to 'New' state if you have required data.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.