Mirantis OpenStack

Cinder leaks on update metadata operation

Series 10.0.x
Bug #1539155

Bug #1539155 reported by Nikita Koshikov on 2016-01-28

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
Mirantis OpenStack	Status tracked in 10.0.x
10.0.x	Fix Committed	Medium	Yuriy Nesenenko	Mirantis OpenStack 10.0
7.0.x	Won't Fix	Medium	MOS Cinder	Mirantis OpenStack 7.0-updates
8.0.x	Won't Fix	Medium	MOS Cinder	Mirantis OpenStack 8.0
9.x	Won't Fix	Medium	Yuriy Nesenenko	Mirantis OpenStack 9.0

Bug Description

This bug was found while running rally:

{
  "CinderVolumes.modify_volume_metadata": [
    {
      "runner": {
        "duration": 1800,
        "type": "constant_for_duration",
        "concurrency": 10
      },
      "context": {
        "volumes": {
          "volumes_per_tenant": 1,
          "size": 4
        }
      }
    }
  ]
}

The result can be seen here http://snag.gy/7vxQS.jpg

MOS Package versions:
cinder-api 1:2015.1.1-1
cinder-common 1:2015.1.1-1
cinder-schedul 1:2015.1.1-1
cinder-volume 1:2015.1.1-1

See original description

Tags:

Ivan Kolodyazhny (e0ne) on 2016-01-28

affects:

fuel → mos

Revision history for this message

Nikita Koshikov (nkoshikov) wrote on 2016-01-28:

cinder.conf Edit (1.6 KiB, text/plain)

Cinder.conf

description:

updated

Revision history for this message

Roman Podoliaka (rpodolyaka) wrote on 2016-01-29:

Please provide more details here: what leaks? It's clear from the graphs that Cinder slows down with time, but it's not clear how important this is and what breaks.

tags:

added: area-cinder

Revision history for this message

Roman Podoliaka (rpodolyaka) wrote on 2016-01-29:

No longer fixing Medium bugs in 8.0, closing as Won't Fix

Revision history for this message

Roman Podoliaka (rpodolyaka) wrote on 2016-01-29:

Feel free to raise the priority if *really* needed. Now it seems like a improvement request to me, not broken functionality.

Roman Podoliaka (rpodolyaka) on 2016-01-29

Changed in mos:
status:	New → Confirmed
status:	Confirmed → Incomplete
importance:	Undecided → Medium
assignee:	nobody → Nikita Koshikov (nkoshikov)
milestone:	none → 9.0

Revision history for this message

Nikita Koshikov (nkoshikov) wrote on 2016-01-29:

I believe the problem somewhere near cinder-api->sqlalchemy->db code, because straces shows lots of db activity and than delay before new poll(). Maybe this is by design.

Logs and strace files provided privately to mos-cinder team.

Vitaly Sedelnik (vsedelnik) on 2016-02-12

tags:

added: wontfix-low

Revision history for this message

Yuriy Nesenenko (ynesenenko) wrote on 2016-02-16:

Possible solution of this problem appears in https://review.openstack.org/#/c/279202/
To reduce the number of API requests and DB queries it would be better if volume metadata keys were deleted with a single request.

Dina Belova (dbelova) on 2016-03-02

Changed in mos:
status:	Incomplete → Confirmed

Revision history for this message

Bug Checker Bot (bug-checker) wrote on 2016-03-28: Autochecker

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

expected result

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

tags:

added: need-info

Revision history for this message

Ivan Kolodyazhny (e0ne) wrote on 2016-04-04:

We're working on a fix for Newton/MOS 10. Is customer need this, we can provide custom package or patch for MOS9 or earlier

tags:	added: move-to-10.0
Changed in mos:
assignee:	Nikita Koshikov (nkoshikov) → nobody

Revision history for this message

Nikita Koshikov (nkoshikov) wrote on 2016-04-05:

Thanks, for now customer is using 7.0 and I'm not sure - regarding their plans for upgrade. As an option - custom package sounds good, but we need to confirm that they will use new code.

Fuel Devops McRobotson (fuel-devops-robot) on 2016-04-12