Stream fanout queues segment size

Bug #2089845 reported by Maximilian Stinsky
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.messaging
New
Undecided
Unassigned

Bug Description

Greetings,

with the release of 2024.1 it is possible to use stream queues instead of fanout queues with the option `rabbit_stream_fanout`.

While testing the new feature I stumbled over the question if it would make sense to define a way smaller stream segment size.

The default segment size of a stream is 500mb.
`x-max-length-bytes` and `x-max-age` are only first applied when the first segment of a stream is full.
That means, for each stream queue, which is enabled in openstack, we will have at least 500mb segment files.

For now I implemented a stream policy directly in rabbitmq to decrease the x-stream-max-segment-size-bytes drastically to 5mb, I am not 100% sure what a good value would be, but transient messages in openstack should be so short lived that most messages are not valid for more than a could of seconds.

As x-stream-max-segment-size-bytes is only used when the stream queue is first created I think it would make sense for oslo.messaging to set a sane default value way lower than 500mb and make it configurable via a config option for any openstack operator to use.

Best Regards
Max

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.