Juniper Openstack

Setting internal_vip to same as external_vip causes high rate of vrrp pkt storm

Bug #1649457 reported by Vedamurthy Joshi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.1
Fix Committed
High
Ignatious Johnson Christopher
Juniper Openstack r3.1.2.0
R3.2
Fix Committed
High
Ignatious Johnson Christopher
Juniper Openstack r3.2.1.0
Trunk
Fix Committed
High
Ignatious Johnson Christopher
Juniper Openstack r4.0.0.0-fcs "r4.0.0.0"

Bug Description

R3.0.2.x Build 13

It is possible that users might set internal_vip to be same as external_vip in env.ha section of testbed.py(when there is no separate ctrl/data network)

It was seen then that keepalived.conf ends up with duplicate VIP config. keepalived starts to generate a very high rate of vrrp advertisements (~15K per sec!) and can cause problems in the network.

We should just skip adding duplicate vips, if any.

vrrp_instance INTERNAL_10_204_216_184 {
        interface p1p1
        state MASTER
        preempt_delay 7
        garp_master_delay 5
        garp_master_repeat 3
        garp_master_refresh 1
        advert_int 1
        virtual_router_id 156
        vmac_xmit_base
        priority 99
        virtual_ipaddress {
                10.204.216.184/24 dev p1p1
        }
        track_script {
                chk_haproxy_INTERNAL_10_204_216_184
        }

        track_script {
            chk_ctrldatanet_INTERNAL_10_204_216_184
        }
        track_interface {
            p1p1
        }
}

vrrp_instance EXTERNAL_10_204_216_184 {
        interface p1p1
        state MASTER
        preempt_delay 7
        garp_master_delay 5
        garp_master_repeat 3
        garp_master_refresh 1
        advert_int 1
        virtual_router_id 156
        vmac_xmit_base
        priority 99
        virtual_ipaddress {
                10.204.216.184/24 dev p1p1
        }
        track_script {
                chk_haproxy_EXTERNAL_10_204_216_184
        }

        track_script {
            chk_ctrldatanet_EXTERNAL_10_204_216_184
        }
        track_interface {
            p1p1
        }
}

-----

15:30:14.250933 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.252026 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.253128 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.254231 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.255351 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.256460 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.257561 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.258686 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.259775 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.260874 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.261976 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.263078 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.264210 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.265323 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.266425 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.267527 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.268629 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.269748 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.270851 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.271948 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.273050 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.274153 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.275272 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.276374 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.277492 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.278585 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.279711 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.280813 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.281915 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.283016 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.284103 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.285194 00:25:90:c3:0a:f2 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 54: 10.204.216.58 > 224.0.0.18: VRRPv2, Advertisement, vrid 156, prio 99, authtype none, intvl 1s, length 20
15:30:14.286313 00:25:90:c3:0a:f2 > 0

Tags: provisioning
Revision history for this message
Ignatious Johnson Christopher (ijohnson-x) wrote :

Section "Supported Cluster Topologies for High Availability " in http://www.juniper.net/techpubs/en_US/contrail2.1/topics/concept/high-avail-support.html

Customers should not provide both the VIP's in the testbed.py

However it will be useful to compare the control_data section and env.ha section and abort provisioning in the pre_check task will be helpful.

Ignatious Johnson Christopher (ijohnson-x)
Changed in juniperopenstack:
milestone: none → r4.0
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/27171
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/27456
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/27487
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/27171
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/211370c9c499137641ffdf78e8d3c7c298d20087
Submitter: Zuul (<email address hidden>)
Branch: master

commit 211370c9c499137641ffdf78e8d3c7c298d20087
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Dec 12 22:34:40 2016 -0800

Section "Supported Cluster Topologies for High Availability in
http://www.juniper.net/techpubs/en_US/contrail2.1/topics/concept/high-avail-support.html

Ensurind the above supported topologies by comparing the control_data section/env.ha
section and abort provisioning in the pre_check task.

Change-Id: Ie8efa4802c275fcfada7d0b577345019376eddd8
Closes-Bug: 1649457

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/27456
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/ad6c942cab8b69c1c54a2b58e8dd9f3d586f9761
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit ad6c942cab8b69c1c54a2b58e8dd9f3d586f9761
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Dec 12 22:34:40 2016 -0800

Section "Supported Cluster Topologies for High Availability in
http://www.juniper.net/techpubs/en_US/contrail2.1/topics/concept/high-avail-support.html

Ensurind the above supported topologies by comparing the control_data section/env.ha
section and abort provisioning in the pre_check task.

Change-Id: Ie8efa4802c275fcfada7d0b577345019376eddd8
Closes-Bug: 1649457

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/27487
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/66079743239c069e4a00d912193f86d3399c36c8
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 66079743239c069e4a00d912193f86d3399c36c8
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Dec 12 22:34:40 2016 -0800

Section "Supported Cluster Topologies for High Availability in
http://www.juniper.net/techpubs/en_US/contrail2.1/topics/concept/high-avail-support.html

Ensurind the above supported topologies by comparing the control_data section/env.ha
section and abort provisioning in the pre_check task.

Change-Id: Ie8efa4802c275fcfada7d0b577345019376eddd8
Closes-Bug: 1649457

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.