Bug #1281148 “QPID reconnection delay can't be configured” : Bugs : oslo.messaging

Flavio Percoco (flaper87) on 2014-02-17

description:	updated
Changed in oslo.messaging:
importance:	Undecided → High
assignee:	nobody → Flavio Percoco (flaper87)
milestone:	none → icehouse-3

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-17: Fix proposed to oslo.messaging (master)

#1

Fix proposed to branch: master
Review: https://review.openstack.org/74080

Changed in oslo.messaging:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-17: Fix proposed to oslo-incubator (stable/havana)

#2

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/74087

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-18: Fix proposed to oslo.messaging (master)

#3

Fix proposed to branch: master
Review: https://review.openstack.org/74315

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-26: Fix merged to oslo.messaging (master)

#4

Reviewed: https://review.openstack.org/74315
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=dc1f984dacced6bbd56b4d97847be8a39b61036e
Submitter: Jenkins
Branch: master

commit dc1f984dacced6bbd56b4d97847be8a39b61036e
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
Closes-bug: #1281148

Changed in oslo.messaging:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-26: Fix proposed to oslo-incubator (stable/havana)

#5

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/76458

Ben Nemec (bnemec) on 2014-03-05

Changed in oslo:
status:	New → Triaged
importance:	Undecided → High

Thierry Carrez (ttx) on 2014-03-06

Changed in oslo.messaging:
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-10: Fix proposed to oslo-incubator (master)

#6

Fix proposed to branch: master
Review: https://review.openstack.org/79304

Changed in oslo:
assignee:	nobody → Flavio Percoco (flaper87)
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-10: Fix merged to oslo-incubator (master)

#7

Reviewed: https://review.openstack.org/79304
Committed: https://git.openstack.org/cgit/openstack/oslo-incubator/commit/?id=8b628d1e024f787dbb93d508117d9148388c0590
Submitter: Jenkins
Branch: master

commit 8b628d1e024f787dbb93d508117d9148388c0590
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Closes-bug: #1281148
Change-Id: I537015f452eb770acba41fdedfe221628f52a920

Changed in oslo:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-17: Fix proposed to neutron (master)

#8

Fix proposed to branch: master
Review: https://review.openstack.org/80998

Changed in neutron:
assignee:	nobody → Ihar Hrachyshka (ihar-hrachyshka)
status:	New → In Progress

Alan Pevec (apevec) on 2014-03-25

Changed in nova:
status:	New → Invalid
Changed in cinder:
status:	New → Invalid

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-25: Fix merged to oslo-incubator (stable/havana)

#9

Reviewed: https://review.openstack.org/76458
Committed: https://git.openstack.org/cgit/openstack/oslo-incubator/commit/?id=ad4dfa7171ac7da1c0fd1020f57f3430fa3bb056
Submitter: Jenkins
Branch: stable/havana

commit ad4dfa7171ac7da1c0fd1020f57f3430fa3bb056
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Closes-bug: #1281148

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
(cherry picked from commit 8b628d1e024f787dbb93d508117d9148388c0590)

Alan Pevec (apevec) on 2014-03-25

Changed in neutron:
importance:	Undecided → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-25: Fix proposed to neutron (stable/havana)

#10

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/82779

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-25:

#11

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/82786

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-27: Fix merged to neutron (stable/havana)

#12

Reviewed: https://review.openstack.org/82786
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7871486133b67be2b1782a67d9d018747cb18571
Submitter: Jenkins
Branch: stable/havana

commit 7871486133b67be2b1782a67d9d018747cb18571
Author: Ihar Hrachyshka <email address hidden>
Date: Tue Mar 25 12:13:23 2014 +0100

Use a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

(the patch is synced from oslo-incubator)

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
Closes-bug: #1281148

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-28: Fix proposed to cinder (stable/havana)

#13

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/83686

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-28: Fix proposed to nova (stable/havana)

#14

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/83689

Adam Gandelman (gandelman-a) on 2014-03-28

Changed in keystone:
assignee:	nobody → Flavio Percoco (flaper87)
importance:	Undecided → High
milestone:	none → 2013.2.3
status:	New → In Progress
assignee:	Flavio Percoco (flaper87) → nobody
status:	In Progress → Invalid
importance:	High → Undecided

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-28: Fix merged to cinder (stable/havana)

#15

Reviewed: https://review.openstack.org/83686
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=c5cdf8261d9f7fb42ca59a12554d1c0ed9cb0a7e
Submitter: Jenkins
Branch: stable/havana

commit c5cdf8261d9f7fb42ca59a12554d1c0ed9cb0a7e
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Closes-bug: #1281148

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
(cherry picked from commit 8b628d1e024f787dbb93d508117d9148388c0590)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-28: Fix merged to keystone (stable/havana)

#16

Reviewed: https://review.openstack.org/83738
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=a96d1a44bc0f074729c312e5c2a0f0875edf1765
Submitter: Jenkins
Branch: stable/havana

commit a96d1a44bc0f074729c312e5c2a0f0875edf1765
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Closes-bug: #1281148

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
(cherry picked from commit 8b628d1e024f787dbb93d508117d9148388c0590)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-29: Fix merged to nova (stable/havana)

#17

Reviewed: https://review.openstack.org/83689
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ecb058de339da8ee828b629c072f4dbb8541fc74
Submitter: Jenkins
Branch: stable/havana

commit ecb058de339da8ee828b629c072f4dbb8541fc74
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Closes-bug: #1281148

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
(cherry picked from commit 8b628d1e024f787dbb93d508117d9148388c0590)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-31: Fix proposed to ceilometer (master)

#18

Fix proposed to branch: master
Review: https://review.openstack.org/84090

Changed in ceilometer:
assignee:	nobody → Flavio Percoco (flaper87)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-31: Fix merged to neutron (master)

#19

Reviewed: https://review.openstack.org/80998
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=665222b38b7580a30f50c1fef78feebcae666f40
Submitter: Jenkins
Branch: master

commit 665222b38b7580a30f50c1fef78feebcae666f40
Author: Ihar Hrachyshka <email address hidden>
Date: Mon Mar 17 14:18:28 2014 +0100

Synced rpc and gettextutils modules from oslo-incubator

    The main reason for sync is to get the following oslo-rpc fixes in Neutron:
    * I537015f452eb770acba41fdedfe221628f52a920 (reduces delays when reconnecting
      to Qpid in HA deployments)
    * Ia148baa6e1ec632789ac3621c85173c2c16f3918 (fixed HA failover, Qpid part)
    * I67923cb024bbd143edc8edccf35b9b400df31eb3 (fixed HA failover, RabbitMQ part)

Latest oslo-incubator commit at the moment of sync:
* 2eab986ef3c43f8d1e25065e3cbc1307860c25c7

    Change-Id: I2f5bb0d195e050f755ecdbf06a6bbed587a04fbe
    Closes-Bug: 1281148
    Closes-Bug: 1261631

Changed in neutron:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-31: Fix merged to ceilometer (master)

#20

Reviewed: https://review.openstack.org/84090
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=93471a2f3ee1bf4d41fa1a21375eaba3942b003a
Submitter: Jenkins
Branch: master

commit 93471a2f3ee1bf4d41fa1a21375eaba3942b003a
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Closes-bug: #1281148

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
(cherry picked from commit 8b628d1e024f787dbb93d508117d9148388c0590)

Changed in ceilometer:
status:	In Progress → Fix Committed

Thierry Carrez (ttx) on 2014-04-01

Changed in neutron:
milestone:	none → icehouse-rc1
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-04-02: Fix merged to ceilometer (stable/havana)

#21

Reviewed: https://review.openstack.org/83739
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=4ffeeadc5caef1ad68c3dbc3f5eec5f74788391b
Submitter: Jenkins
Branch: stable/havana

commit 4ffeeadc5caef1ad68c3dbc3f5eec5f74788391b
Author: Flavio Percoco <email address hidden>
Date: Tue Feb 18 10:56:21 2014 +0100

User a more accurate max_delay for reconnects

    In an HA deployment, a 60 seconds delay between reconnects can be quite
    problematic. This patch changes the delay calculation by setting the max
    delay to 5s and by changing the way it is increased.

    Unfortunately, this is one of the places where both our main drivers are
    not consistent. Rabbit's driver uses configuration parameters for this
    whereas qpid's driver has never had one. However, I would prefer not
    adding configuration paremeters to qpid's driver for the following
    reasons:

        1. Most of OpenStack services depend on the messaging layer, hence
        they need it to be available. A 5s delay seems to be reasonable and
        I could argue the need of tune it further. Although so frequent
        reconnects can add load to the network, that wouldn't be the main
        issue if one of the brokers go down.
        2. We're trying to move away from configuration options towards using
        transport URL. This path is still not clear and I would
        prefer avoiding adding new options until we clear it out.

Closes-bug: #1281148

Change-Id: I537015f452eb770acba41fdedfe221628f52a920
(cherry picked from commit 93471a2f3ee1bf4d41fa1a21375eaba3942b003a)

Adam Gandelman (gandelman-a) on 2014-04-03

Changed in keystone:
milestone:	2013.2.3 → none

Thierry Carrez (ttx) on 2014-04-04

Changed in oslo:
milestone:	none → icehouse-rc1
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2014-04-17

Changed in oslo:
milestone:	icehouse-rc1 → 2014.1

Thierry Carrez (ttx) on 2014-04-17

Changed in oslo.messaging:
milestone:	icehouse-3 → 1.3.0

Thierry Carrez (ttx) on 2014-04-17

Changed in neutron:
milestone:	icehouse-rc1 → 2014.1

Eoghan Glynn (eglynn) on 2014-05-27

Changed in ceilometer:
importance:	Undecided → High
milestone:	none → juno-1

Thierry Carrez (ttx) on 2014-06-12

Changed in ceilometer:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2014-10-16

Changed in ceilometer:
milestone:	juno-1 → 2014.2

oslo.messaging

QPID reconnection delay can't be configured

Bug Description

Other bug subscribers

Remote bug watches

	Status	Importance	Assigned to	Milestone
Ceilometer	Fix Released	High	Flavio Percoco	Ceilometer 2014.2 "juno"
Havana	Fix Released	Undecided	Flavio Percoco	Ceilometer 2013.2.3
Cinder	Invalid	Undecided	Unassigned
Havana	Fix Released	High	Flavio Percoco	Cinder 2013.2.3
OpenStack Compute (nova)	Invalid	Undecided	Unassigned
Havana	Fix Released	High	Flavio Percoco	OpenStack Compute (nova) 2013.2.3
OpenStack Identity (keystone)	Invalid	Undecided	Unassigned
Havana	Fix Released	High	Flavio Percoco	OpenStack Identity (keystone) 2013.2.3
neutron	Fix Released	High	Ihar Hrachyshka	neutron 2014.1 "icehouse"
Havana	Fix Released	High	Ihar Hrachyshka	neutron 2013.2.3
oslo-incubator	Fix Released	High	Flavio Percoco	oslo-incubator 2014.1 "icehouse"
Havana	Fix Committed	High	Flavio Percoco
oslo.messaging	Fix Released	High	Flavio Percoco	oslo.messaging 1.3.0