Cinder

LVM should clone/migrate efficiently

Bug #1224671 reported by Eric Harney
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Medium
Mitsuhiro Tanino
Cinder newton-1 "n1"
Declined for Kilo by Eric Harney

Bug Description

ThinLVM uses volutils.copy_volume to copy data between LVs for clone and migrate operations. In the case of clone this should be done with an LVM operation that's smart enough to not copy the unused segments of the LV. In the case of migrate, if for some reason we're migrating to the same VG/LVM node we should optimize this as well.

Currently this (I believe) results in Thin LVs that are 100% populated when cloning or migrating from other Thin LVs.

See original description
Tags: drivers lvm
John Griffith (john-griffith)
summary: - ThinLVM should clone/migrate efficiently
+ LVM should clone efficiently
Changed in cinder:
status: New → Triaged
importance: Low → Medium
description: updated
Revision history for this message
Eric Harney (eharney) wrote : Re: LVM should clone efficiently

Also consider the possibility of using dd conv=sparse for remote migration of ThinLVM, which should skip writing empty blocks, saving time and space.

John Griffith (john-griffith)
Changed in cinder:
milestone: havana-rc1 → none
Revision history for this message
John Griffith (john-griffith) wrote :

Fixing up the LVM-Thin version shoudl be a priority in I-1, finding a better solution for LVM-Thick would be good too, but not as high priority IMO.

Changed in cinder:
importance: Medium → Wishlist
importance: Wishlist → Medium
milestone: none → icehouse-1
John Griffith (john-griffith)
Changed in cinder:
assignee: nobody → John Griffith (john-griffith)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/59581

John Griffith (john-griffith)
Changed in cinder:
milestone: icehouse-1 → icehouse-2
Thierry Carrez (ttx)
Changed in cinder:
milestone: icehouse-2 → icehouse-3
Thierry Carrez (ttx)
Changed in cinder:
milestone: icehouse-3 → icehouse-rc1
John Griffith (john-griffith)
Changed in cinder:
milestone: icehouse-rc1 → next
Mike Perez (thingee)
tags: added: drivers lvm
Eric Harney (eharney)
Changed in cinder:
status: In Progress → Confirmed
Mitsuhiro Tanino (mitsuhiro-tanino)
Changed in cinder:
assignee: John Griffith (john-griffith) → Mitsuhiro Tanino (mitsuhiro-tanino)
Mitsuhiro Tanino (mitsuhiro-tanino)
summary: - LVM should clone efficiently
+ LVM should clone/migrate efficiently
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/182473

Changed in cinder:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/182473
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=4e4a54d41ca8801f35d9988d0b8742bf354df049
Submitter: Jenkins
Branch: master

commit 4e4a54d41ca8801f35d9988d0b8742bf354df049
Author: Mitsuhiro Tanino <email address hidden>
Date: Fri May 8 15:00:03 2015 -0400

    LVM: Support efficient data copy using "dd" for create_cloned_volume

    The create_cloned_volume uses dd command for data copy, but the
    copy always copy full blocks even if the source data contains
    many null or zero blocks. When we use thin provisioned LVM,
    blocks are not pre-allocated, so unused region returns zero.
    If we copy full block for destination volume, unnecessary blocks
    will be allocated and the usage will be 100%.

    The dd command has conv=sparse option in order to copy data more
    efficiently. This patch enables conv=sparse option as an argument
    of dd command for create_cloned_volume when we use thin provisioned
    LVM.

    [NOTE]
    This patch only enables conv=sparse parameter of dd command for
    create_cloned_volume() path of LVM driver. There are some places
    using dd in Cinder, but we should carefully consider to apply
    this parameter for other places because misuse of this parameter
    causes security issues or data corruptions. Also we DO NOT use this
    parameter for volume wiping case because the volume is not cleared
    at all.

    Here are some results for this option.

    - Without conv=sparse option
      LV VG Attr LSize Pool Origin Data%
      vg1-pool vg1 twi-a-tz-- 3.80g 31.45
      volume-clone vg1 Vwi-a-tz-- 1.00g vg1-pool 100.00
      volume-source vg1 Vwi-a-tz-- 1.00g vg1-pool 19.53

    - With conv=sparse option
      LV VG Attr LSize Pool Origin Data%
      vg1-pool vg1 twi-a-tz-- 3.80g 10.28
      volume-clone vg1 Vwi-a-tz-- 1.00g vg1-pool 19.53
      volume-source vg1 Vwi-a-tz-- 1.00g vg1-pool 19.53

    Change-Id: I743f823ca38529b12301a89308d1d406aa3fa45f
    Closes-bug: #1224671

Changed in cinder:
status: In Progress → Fix Committed
Revision history for this message
Sean McGinnis (sean-mcginnis) wrote :

Automatically unassigning due to inactivity.

Changed in cinder:
assignee: Mitsuhiro Tanino (mitsuhiro-tanino) → nobody
Mitsuhiro Tanino (mitsuhiro-tanino)
Changed in cinder:
assignee: nobody → Mitsuhiro Tanino (mitsuhiro-tanino)
status: Fix Committed → Fix Released
Revision history for this message
Eric Harney (eharney) wrote :

This was released in 7.0.0.

Revision history for this message
Eric Harney (eharney) wrote :

This was fixed in Liberty, but I can't target the bug there any longer, so targeting it to Newton-1.

Changed in cinder:
milestone: next → newton-1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.