kolla-ansible

ironic-prometheus-exporter causes Docker mount havoc

  1. Series caracal
  2. Bug #2062401
Bug #2062401 reported by Michal Nasiadka
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Released
High
Michal Nasiadka
Bobcat
Fix Released
High
Unassigned
Caracal
Fix Released
High
Unassigned
Dalmatian
Fix Released
High
Michal Nasiadka

Bug Description

After some time of running you get this:

/dev/mapper/rl-var on /var/lib/docker/volumes/ironic/_data/metrics type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
overlay on /var/lib/docker/overlay2/45470a4eccc17bce0529f27ca53ad858912b760e1323b1b38b24db42acced940/merged type overlay (rw,relatime,seclabel,lowerdir=/var/lib/docker/overlay2/l/4YYYRSNXVSH3HAVFMRM5PKXVN2:/var/lib/docker/overlay2/l/WQTW7VIVPA7MB2OMPJVIB2Z6DD:/var/lib/docker/overlay2/l/G6XRI6AT7O264ZEIMFRUQIIXCY:/var/lib/docker/overlay2/l/AYY663FWIKUZ2G5RJMLPSDY2XK:/var/lib/docker/overlay2/l/7NVNFTRSQUXYLQ4D4DCCM7UMBM:/var/lib/docker/overlay2/l/SEFLTAD6IDMW6KXBEEBEUF55RU:/var/lib/docker/overlay2/l/UWRXZAWCPBF3BF75JLEU7SM3YG:/var/lib/docker/overlay2/l/N6NTYI6LPCAFKIIM4QP6NQWLXZ:/var/lib/docker/overlay2/l/RBX3TQ6W3TWXKI5PRPQCKPDYFF:/var/lib/docker/overlay2/l/U76VGJVU23Q5RDRZHSTYIVEFOR:/var/lib/docker/overlay2/l/Q7SWZWJKDRRMAO4ZB6E26DSOPC:/var/lib/docker/overlay2/l/SYO42ZP6X6VNA5YVGXL52QO6UZ:/var/lib/docker/overlay2/l/BKVY62HLJJ5IZNRBADH4HNHWSA:/var/lib/docker/overlay2/l/MZ3BVICMMQTJX7XFQOAEGTDM5F:/var/lib/docker/overlay2/l/E52JCBKNQ7W22ME63JWEOC5J5R:/var/lib/docker/overlay2/l/EWQLVAICDUKV6GS5QEL4QSFPWA:/var/lib/docker/overlay2/l/FQBSYQTTOSRKTYEAW4KSQWBCSG:/var/lib/docker/overlay2/l/3GXNZ4BBQGAFK7XXPPYZKQQI7T:/var/lib/docker/overlay2/l/UGYVZCJ262ODJRJDLW5EBTS6WK:/var/lib/docker/overlay2/l/RSMM3VLIKQQE5RV57WZOPPMQDL:/var/lib/docker/overlay2/l/UY2HU5CW34WZRZEDEA3ZI7FHPR:/var/lib/docker/overlay2/l/CNFGWTFBSBKLHWC5W4OJRZOWVE:/var/lib/docker/overlay2/l/O5A4QQ42FVPV4H4HIKEHFZSEVC:/var/lib/docker/overlay2/l/TIGHYPXJV6QVMY3FWS2B56VVH6:/var/lib/docker/overlay2/l/RTWRXS6QIYJINGWASER5K5TIAO:/var/lib/docker/overlay2/l/2BQB5KMQANJJKVOVBSSCGX4TEM:/var/lib/docker/overlay2/l/A6NLZNEKR2FTLMEFX6YLPKQZ34:/var/lib/docker/overlay2/l/XKCN6ZF2K3PSWTYD3EWNUMZYQC:/var/lib/docker/overlay2/l/6Q2LNP4PWH7AU5GXBYGOXEFVXU:/var/lib/docker/overlay2/l/YO3OWQTG3646EV2BWC32DYFNCB:/var/lib/docker/overlay2/l/S5SME4FIVZW52X7YTFXGPHIWLZ:/var/lib/docker/overlay2/l/A7BZ6VKZQVLRYH3I2G4QXEMFK2:/var/lib/docker/overlay2/l/HONYMYEWJBMJ6NUKN3TC7RI4IP,upperdir=/var/lib/docker/overlay2/45470a4eccc17bce0529f27ca53ad858912b760e1323b1b38b24db42acced940/diff,workdir=/var/lib/docker/overlay2/45470a4eccc17bce0529f27ca53ad858912b760e1323b1b38b24db42acced940/work)

Michal Nasiadka (mnasiadka)
Changed in kolla-ansible:
importance: Undecided → High
assignee: nobody → Michal Nasiadka (mnasiadka)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/914039
Committed: https://opendev.org/openstack/kolla-ansible/commit/b391cdcf95e23c666cc31b9c706f12d8820cf146
Submitter: "Zuul (22348)"
Branch: master

commit b391cdcf95e23c666cc31b9c706f12d8820cf146
Author: Michal Nasiadka <email address hidden>
Date: Sun Mar 24 09:13:05 2024 +0100

    ironic: mount metrics in conductor only when exporter is enabled

    Related-Bug: #2062401

    Change-Id: I2c058e0fe56b32fce36e1c9f637c12fe5950ff21

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to kolla-ansible (stable/2024.1)

Related fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/919512

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to kolla-ansible (stable/2023.2)

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/919513

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to kolla-ansible (stable/2024.1)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/919512
Committed: https://opendev.org/openstack/kolla-ansible/commit/bcb330e5d3fa12f282a9cc719f54ae630875a22d
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit bcb330e5d3fa12f282a9cc719f54ae630875a22d
Author: Michal Nasiadka <email address hidden>
Date: Sun Mar 24 09:13:05 2024 +0100

    ironic: mount metrics in conductor only when exporter is enabled

    Related-Bug: #2062401

    Change-Id: I2c058e0fe56b32fce36e1c9f637c12fe5950ff21
    (cherry picked from commit b391cdcf95e23c666cc31b9c706f12d8820cf146)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to kolla-ansible (stable/2023.2)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/919513
Committed: https://opendev.org/openstack/kolla-ansible/commit/ce1a1d15f155e7c6e2bbb29ae01ffd29b15658c8
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit ce1a1d15f155e7c6e2bbb29ae01ffd29b15658c8
Author: Michal Nasiadka <email address hidden>
Date: Sun Mar 24 09:13:05 2024 +0100

    ironic: mount metrics in conductor only when exporter is enabled

    Related-Bug: #2062401

    Change-Id: I2c058e0fe56b32fce36e1c9f637c12fe5950ff21
    (cherry picked from commit b391cdcf95e23c666cc31b9c706f12d8820cf146)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/938603

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/938603
Committed: https://opendev.org/openstack/kolla-ansible/commit/904fae2a91a9536841f252847103dacfa27d81ce
Submitter: "Zuul (22348)"
Branch: master

commit 904fae2a91a9536841f252847103dacfa27d81ce
Author: Michal Nasiadka <email address hidden>
Date: Tue Jan 7 18:50:00 2025 +0100

    ironic: Change Prometheus metrics dir

    Closes-Bug: #2062401
    Change-Id: I2f2bdbc9e1c6ad6da4ac7098ddd36143123c3062

Changed in kolla-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/2024.2)

Fix proposed to branch: stable/2024.2
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/939747

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/2024.1)

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/939748

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/2023.2)

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/939768

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/2023.2)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/939768
Committed: https://opendev.org/openstack/kolla-ansible/commit/0e2a6f186e1508087a431b60df223cd39ff1c944
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 0e2a6f186e1508087a431b60df223cd39ff1c944
Author: Michal Nasiadka <email address hidden>
Date: Tue Jan 7 18:50:00 2025 +0100

    ironic: Change Prometheus metrics dir

    Closes-Bug: #2062401
    Change-Id: I2f2bdbc9e1c6ad6da4ac7098ddd36143123c3062
    (cherry picked from commit 904fae2a91a9536841f252847103dacfa27d81ce)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/2024.2)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/939747
Committed: https://opendev.org/openstack/kolla-ansible/commit/d6065fba5168f3b3e19960da231f711fad3a1cda
Submitter: "Zuul (22348)"
Branch: stable/2024.2

commit d6065fba5168f3b3e19960da231f711fad3a1cda
Author: Michal Nasiadka <email address hidden>
Date: Tue Jan 7 18:50:00 2025 +0100

    ironic: Change Prometheus metrics dir

    Closes-Bug: #2062401
    Change-Id: I2f2bdbc9e1c6ad6da4ac7098ddd36143123c3062
    (cherry picked from commit 904fae2a91a9536841f252847103dacfa27d81ce)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/2024.1)

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/939748
Committed: https://opendev.org/openstack/kolla-ansible/commit/b53c6d9707683ee86e2ddaec4e732369fe74e162
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit b53c6d9707683ee86e2ddaec4e732369fe74e162
Author: Michal Nasiadka <email address hidden>
Date: Tue Jan 7 18:50:00 2025 +0100

    ironic: Change Prometheus metrics dir

    Closes-Bug: #2062401
    Change-Id: I2f2bdbc9e1c6ad6da4ac7098ddd36143123c3062
    (cherry picked from commit 904fae2a91a9536841f252847103dacfa27d81ce)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 17.8.0

This issue was fixed in the openstack/kolla-ansible 17.8.0 Bobcat release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 18.5.0

This issue was fixed in the openstack/kolla-ansible 18.5.0 Caracal release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 19.3.0

This issue was fixed in the openstack/kolla-ansible 19.3.0 Dalmatian release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 20.0.0.0rc1

This issue was fixed in the openstack/kolla-ansible 20.0.0.0rc1 Epoxy release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.