queue TTL pattern does not match intended queues

Bug #1939681 reported by Gabriel Samfira
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack RabbitMQ Server Charm
Fix Committed
High
Edward Hope-Morley
Focal
Fix Released
Undecided
Unassigned
Jammy
Fix Released
Undecided
Unassigned

Bug Description

This PR adds the ability for charms to set an expiry policy on queues:

https://opendev.org/openstack/charm-rabbitmq-server/commit/707fa0e093af3b6ab636b65b018c29049f7a51ff

This is especially useful for heat, where restating the service will leave behind old queues, making the count grow considerably over time. However, the pattern used is enclosed in double quotes, and rabbitmq treats the quotes as part of the pattern, which means it will never match. Here is the policy created in rabbitmq:

# rabbitmqctl list_policies -p openstack
Listing policies for vhost "openstack" ...
vhost name pattern apply-to definition priority
openstack HA ^(?!amq\.).* all {"ha-mode":"all","ha-sync-mode":"automatic"} 0
openstack heat_expiry "heat-engine-listener|engine_worker" queues {"expires":3600000} 1
openstack TTL ^(versioned_)?notifications.* queues {"message-ttl":3600000} 1

Notice the quotes surrounding the pattern:

 "heat-engine-listener|engine_worker"

The following diff, makes it work as intended:

diff --git a/hooks/rabbit_utils.py b/hooks/rabbit_utils.py
index f1594c5..be8968a 100644
--- a/hooks/rabbit_utils.py
+++ b/hooks/rabbit_utils.py
@@ -443,7 +443,7 @@ def configure_ttl(vhost, ttlname, ttlreg, ttl):
         return
     rabbitmqctl('set_policy',
                 '{ttlname}'.format(ttlname=ttlname),
- '"{ttlreg}"'.format(ttlreg=ttlreg),
+ '{ttlreg}'.format(ttlreg=ttlreg),
                 '{{"expires":{ttl}}}'.format(ttl=ttl),
                 '--priority', '1',
                 '--apply-to', 'queues',

Revision history for this message
Edward Hope-Morley (hopem) wrote :

confirmed problem exists on master branch and that this change fixes it

Changed in charm-rabbitmq-server:
status: New → Confirmed
assignee: nobody → Edward Hope-Morley (hopem)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-rabbitmq-server (master)
Changed in charm-rabbitmq-server:
status: Confirmed → In Progress
Revision history for this message
Edward Hope-Morley (hopem) wrote :

I'll mark this as high priority since it leads to expiry not being applied and therefore queues piling up and consuming resources in rabbitmq server.

Changed in charm-rabbitmq-server:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-rabbitmq-server (stable/jammy)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-rabbitmq-server (master)

Reviewed: https://review.opendev.org/c/openstack/charm-rabbitmq-server/+/879345
Committed: https://opendev.org/openstack/charm-rabbitmq-server/commit/8c9f68e8aadc3a6786bf42ca250252fed0eef381
Submitter: "Zuul (22348)"
Branch: master

commit 8c9f68e8aadc3a6786bf42ca250252fed0eef381
Author: Edward Hope-Morley <email address hidden>
Date: Mon Apr 3 14:30:34 2023 +0100

    Fix typo in configure ttl code

    Also fixes tox.ini

    Change-Id: Ic4c2d34ff248d5429eb604824e42dbaba6ca2678
    Closes-Bug: #1939681

Changed in charm-rabbitmq-server:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-rabbitmq-server (stable/focal)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-rabbitmq-server (stable/jammy)

Reviewed: https://review.opendev.org/c/openstack/charm-rabbitmq-server/+/880050
Committed: https://opendev.org/openstack/charm-rabbitmq-server/commit/ba80227067f2268589e0b273ef5f783369fb8c8a
Submitter: "Zuul (22348)"
Branch: stable/jammy

commit ba80227067f2268589e0b273ef5f783369fb8c8a
Author: Edward Hope-Morley <email address hidden>
Date: Mon Apr 3 14:30:34 2023 +0100

    Fix typo in configure ttl code

    Also fixes tox.ini

    Change-Id: Ic4c2d34ff248d5429eb604824e42dbaba6ca2678
    Closes-Bug: #1939681
    (cherry picked from commit 8c9f68e8aadc3a6786bf42ca250252fed0eef381)

tags: added: in-stable-jammy
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-rabbitmq-server (stable/focal)

Reviewed: https://review.opendev.org/c/openstack/charm-rabbitmq-server/+/880532
Committed: https://opendev.org/openstack/charm-rabbitmq-server/commit/ddea6a0f4160dbd94074963cb49c8ffc318fd442
Submitter: "Zuul (22348)"
Branch: stable/focal

commit ddea6a0f4160dbd94074963cb49c8ffc318fd442
Author: Edward Hope-Morley <email address hidden>
Date: Mon Apr 3 14:30:34 2023 +0100

    Fix typo in configure ttl code

    Also fixes tox.ini

    Change-Id: Ic4c2d34ff248d5429eb604824e42dbaba6ca2678
    Closes-Bug: #1939681
    (cherry picked from commit 8c9f68e8aadc3a6786bf42ca250252fed0eef381)
    (cherry picked from commit ba80227067f2268589e0b273ef5f783369fb8c8a)

tags: added: in-stable-focal
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.