oslo.messaging

gate-tempest-dsvm-largeops-src-oslo.messaging fail

Bug #1408362 reported by Mehdi Abaakouk
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.messaging
Fix Released
Critical
Mehdi Abaakouk
oslo.messaging 1.6.0
oslo.utils
Fix Released
Critical
Doug Hellmann
oslo.utils 1.1.1

Bug Description

Hi,

gate-tempest-dsvm-largeops-src-oslo.messaging gate job is currently broken.

After investigation, this is due to the strutils.mask_password method used for logging purpose here that take to much time:

https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/common.py#L166

The regex that take too much time:

                      r'([\'"].*?%(key)s[\'"]\s*:\s*u?[\'"]).*?([\'"])',
                      r'([\'"].*?%(key)s[\'"]\s*,\s*\'--?[A-z]+\'\s*,\s*u?[\'"]).*?([\'"])',

During the gate job run we can see the cpu stuck at 100% by nova-conductor, because mask_password try to hide password of ~100 message of 66k.

Cheers,

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.messaging (master)

Fix proposed to branch: master
Review: https://review.openstack.org/145541

Changed in oslo.messaging:
assignee: nobody → Mehdi Abaakouk (sileht)
status: New → In Progress
Mehdi Abaakouk (sileht)
Changed in oslo.messaging:
importance: Undecided → Critical
milestone: none → next-kilo
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.messaging (master)

Reviewed: https://review.openstack.org/145541
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=c18f9f7c611df0e4c8e7e8d2067b31406147d264
Submitter: Jenkins
Branch: master

commit c18f9f7c611df0e4c8e7e8d2067b31406147d264
Author: Mehdi Abaakouk <email address hidden>
Date: Wed Jan 7 17:50:44 2015 +0100

    Don't log each received messages

    oslo.utils.strutils.mask_password take too much time on
    big payload (nova-conductor can receive payload ~ 66k in
    largeops jobs for example).

    This change removes this logging until we make mask_password
    more efficient or we have a smarted the oslo.messaging logging to
    not log everything.

    Change-Id: Ib1f1d70c5cb820e8ff2de10e6064037808ea1f3a
    Closes-bug: #1408362

Changed in oslo.messaging:
status: In Progress → Fix Committed
Doug Hellmann (doug-hellmann)
Changed in oslo.utils:
assignee: nobody → Doug Hellmann (doug-hellmann)
importance: Undecided → Critical
status: New → Fix Committed
milestone: none → next-kilo
Doug Hellmann (doug-hellmann)
Changed in oslo.utils:
status: Fix Committed → Fix Released
Doug Hellmann (doug-hellmann)
Changed in oslo.messaging:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on oslo.messaging (master)

Change abandoned by Mehdi Abaakouk (sileht) (<email address hidden>) on branch: master
Review: https://review.openstack.org/191242

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.