Glance image_cache_max_size is larger than container size

Bug #1403487 reported by Sam Yaple
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openstack-ansible
Fix Released
High
Unassigned
Icehouse
Fix Released
Medium
Jesse Pretorius
Juno
Fix Released
Medium
Jesse Pretorius
Kilo
Fix Released
High
Unassigned
Trunk
Fix Released
High
Unassigned

Bug Description

Currently glance image_cache_max_size is currently set to 10GiB while the containers are being built with only 5GB of space causing the container to get to 100% filesystem usage.

Changed in openstack-ansible:
status: New → Confirmed
importance: Undecided → Medium
milestone: none → 10.1.2
milestone: 10.1.2 → next
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-ansible-deployment (master)

Reviewed: https://review.openstack.org/142433
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=362e727dabd8d118d515208382d5849f01096eec
Submitter: Jenkins
Branch: master

commit 362e727dabd8d118d515208382d5849f01096eec
Author: Sam Yaple <email address hidden>
Date: Wed Dec 17 06:22:02 2014 -0600

    Make image_cache_max_size user configurable

    image_cache_max_size should be less than the size of the container. It defaults
    to 10GiB which is greater than the current default container size for glance of
    5GB.

    Change-Id: I58ad98ba3cf83a63dbe346659148aba53dafd140
    Related-Bug: 1403487

Changed in openstack-ansible:
status: Confirmed → Fix Committed
Changed in openstack-ansible:
assignee: nobody → Sam Yaple (s8m)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-ansible-deployment (juno)

Related fix proposed to branch: juno
Review: https://review.openstack.org/151176

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to os-ansible-deployment (icehouse)

Related fix proposed to branch: icehouse
Review: https://review.openstack.org/151178

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-ansible-deployment (icehouse)

Reviewed: https://review.openstack.org/151178
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=226dcf7c1d780a382bfb6406b7bf02bb01468db3
Submitter: Jenkins
Branch: icehouse

commit 226dcf7c1d780a382bfb6406b7bf02bb01468db3
Author: Sam Yaple <email address hidden>
Date: Wed Dec 17 06:22:02 2014 -0600

    Make image_cache_max_size user configurable

    image_cache_max_size should be less than the size of the container. It defaults
    to 10GiB which is greater than the current default container size for glance of
    5GB.

    Change-Id: I58ad98ba3cf83a63dbe346659148aba53dafd140
    Related-Bug: 1403487
    (cherry picked from commit 362e727dabd8d118d515208382d5849f01096eec)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to os-ansible-deployment (juno)

Reviewed: https://review.openstack.org/151176
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=9e104f979e2ac5e3a54cff6b950a7c0005e5d13d
Submitter: Jenkins
Branch: juno

commit 9e104f979e2ac5e3a54cff6b950a7c0005e5d13d
Author: Sam Yaple <email address hidden>
Date: Wed Dec 17 06:22:02 2014 -0600

    Make image_cache_max_size user configurable

    image_cache_max_size should be less than the size of the container. It defaults
    to 10GiB which is greater than the current default container size for glance of
    5GB.

    Change-Id: I58ad98ba3cf83a63dbe346659148aba53dafd140
    Related-Bug: 1403487
    (cherry picked from commit 362e727dabd8d118d515208382d5849f01096eec)

Revision history for this message
Kevin Carter (kevin-carter) wrote :

In master this still effects os_glance role:

The following needs to be <= the fs size of the container.
https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/roles/os_glance/defaults/main.yml#L91

Revision history for this message
Jimmy McCrory (jimmy-mccrory) wrote :

The cache directory, or even the entire /var/lib/glance directory, could be mounted on the host similarly to how /var/lib/glance/images is here:

https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/os-glance-install.yml#L21-L31

That could resolve this while keeping the upstream default cache size and still allowing users to increase or decrease that value without having to treat the filesize of the glance containers special.

Revision history for this message
Justin Shepherd (jshepher) wrote :

I agree with the above comment. It seems like making sure the cache directory is bind mounted out of the container would be the better path, instead of decreasing the cache size to something small like 5G.

Revision history for this message
Andy McCrae (andrew-mccrae) wrote :

I disagree - I don't think we should be mounting extra directories into a container unless it is absolutely necessary.
We should increase the default glance container size to > 10G (which I though we had done already). In fact that is already the case, the glance container should be 12G on master, we can easily increase this again - NB this is set in the environment file under properties for glance:

I would argue that we shouldn't even be mounting the images directory into the container, although I can see the use case where you remove a container and recreate it - so the images would still exist (but this would only work in an AIO, and file backed glance isn't ideal for multiple glance containers).

Revision history for this message
Andy McCrae (andrew-mccrae) wrote :
Revision history for this message
Jimmy McCrory (jimmy-mccrory) wrote :

Hadn't seen that the container fs size was already increased.
In that case, should be bug be closed for kilo/trunk?

Revision history for this message
Darren Birkett (darren-birkett) wrote :

Separate bug containing master fix:

https://bugs.launchpad.net/openstack-ansible/+bug/1442115

So yes, we can close this for master (and kilo since it was branched from master after this commit)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.