Juniper Openstack

R3.0.3.x - build16 - Centos7.1 agent introspect is failing to list VN

Series r3.0
Bug #1665309

Bug #1665309 reported by Sudheendra Rao on 2017-02-16

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
Juniper Openstack	Status tracked in Trunk
R3.0	Fix Committed	Critical	Sudheendra Rao	Juniper Openstack r3.0.4.0
R3.0.2.x	Fix Committed	Critical	Sudheendra Rao	Juniper Openstack r3.0.2.1
R3.0.3.x	Fix Committed	Critical	Sudheendra Rao	Juniper Openstack r3.0.3.3
R3.1	Fix Committed	Critical	Sudheendra Rao	Juniper Openstack r3.1.3.0
R3.2	Invalid	Critical	Sudheendra Rao	Juniper Openstack r3.2.1.0
Trunk	Invalid	Critical	Sudheendra Rao	Juniper Openstack r4.0.0.0-fcs "r4.0.0.0"

Bug Description

agent introspect failing to list VNs on centos7.1 R3.0.3.x build16 juno.
The sanity scripts are failing while verifying VM in agent due to above problem.

The setup is in failed state: nodeb2.englab.juniper.net (root/c0ntrail123)

The agent introspect to list VN:
http://nodeb2.englab.juniper.net:8085/Snh_VnListReq?name=&uuid=&vxlan_id=&ipam_name=

Agent introspect for interface showing VRF in ERROR state:
http://nodeb2.englab.juniper.net:8085/Snh_ItfReq?name=&type=&uuid=&vn=&mac=&ipv4_address=&ipv6_address=&parent_uuid=&ip_active=&ip6_active=&l2_active=

Test log:

2017-02-16 03:20:07,519 - WARNING - VN default-domain:ctest-TestBasicVMVN-63889369:ctest-ctest-TestBasicVMVN-63889369-34044572 is not seen in agent 10.204.216.33
2017-02-16 03:20:09,521 - DEBUG - Requesting: http://10.204.216.33:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicVMVN-63889369:ctest-ctest-TestBasicVMVN-63889369-34044572
2017-02-16 03:20:09,526 - WARNING - VN default-domain:ctest-TestBasicVMVN-63889369:ctest-ctest-TestBasicVMVN-63889369-34044572 is not seen in agent 10.204.216.33
2017-02-16 03:20:11,529 - DEBUG - Requesting: http://10.204.216.33:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicVMVN-63889369:ctest-ctest-TestBasicVMVN-63889369-34044572
2017-02-16 03:20:11,532 - WARNING - VN default-domain:ctest-TestBasicVMVN-63889369:ctest-ctest-TestBasicVMVN-63889369-34044572 is not seen in agent 10.204.216.33
2017-02-16 03:20:13,535 - DEBUG - Requesting: http://10.204.216.33:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicVMVN-63889369:ctest-ctest-TestBasicVMVN-63889369-34044572
2017-02-16 03:20:13,540 - WARNING - VN default-domain:ctest-TestBasicVMVN-63889369:ctest-ctest-TestBasicVMVN-63889369-34044572 is not seen in agent 10.204.216.33
2017-02-16 03:20:13,540 - ERROR - VM ctest-vm_add_delete-71838320 verification in Agent failed

Tags:

Revision history for this message

Senthilnathan Murugappan (msenthil) wrote on 2017-02-16:

I don’t see the VM subscribe messages been sent from Agent to controller.
I do see few “ConfigPortInfo: Interface does not exist in OS controller/src/vnsw/agent/port_ipc/port_ipc_handler.cc 275” at http://nodeb2.englab.juniper.net:8085/Snh_SandeshTraceRequest?x=Config however http://nodeb2.englab.juniper.net:8085/Snh_ItfReq?name=&type=&uuid=&vn=&mac=&ipv4_address=&ipv6_address=&parent_uuid=&ip_active=&ip6_active=&l2_active= has os if_index set so not sure whether it’s the real reason.

Jeba Paulaiyan (jebap) on 2017-02-22

tags:

added: vrouter
removed: config

Revision history for this message

Hari Prasad Killi (haripk) wrote on 2017-02-22:

commit https://github.com/Juniper/contrail-provisioning/commit/08ddf9221b6083fcebc58af01dfa4dfc856ca1e7 added a supervisor-vrouter restart in contrail_provisioning/compute/scripts/compute-server-setup.sh, which is the reason vrouter kernel module is getting loaded.

Revision history for this message

Hari Prasad Killi (haripk) wrote on 2017-02-23:

Earlier comment is not for this bug, please ignore.

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-02-23: [Review update] R3.0.3.x

Review in progress for https://review.opencontrail.org/29065
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-02-24: A change has been merged

Reviewed: https://review.opencontrail.org/29065
Committed: http://github.org/Juniper/contrail-controller/commit/e522093fdf6c18260a7e742d05f96b2e601ce3b5
Submitter: Zuul (<email address hidden>)
Branch: R3.0.3.x

commit e522093fdf6c18260a7e742d05f96b2e601ce3b5
Author: ashoksingh <email address hidden>
Date: Thu Feb 23 17:49:27 2017 +0530

Fix issue of VM not getting deleted from Agent

Nova repo is absent in R3.0.3.x/centos71/manifest-juno.xml. This repo has changes to invoke/use Agent’s REST API for port Adds/deletes.
Until we have this repo in all manifest files of R3.0.3.x branches continue to use Thrift instead of REST.

This reverts the following commits
https://github.com/Juniper/contrail-controller/commit/8c1eacfb85bb878f695c67bc6ef3ca6ed9e8f127
https://github.com/Juniper/contrail-controller/commit/6e3d4a8ac336f78baaa46877ad833dc9f5752707
https://github.com/Juniper/contrail-controller/commit/de72bafd363f96048ece41785b0b6f55e64f3a0d (only partial revert)

Change-Id: Ia8049ada179df241c9782aebc593857facfd2f5a
Closes-Bug: #1665309

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-02-24: [Review update] R3.1

Review in progress for https://review.opencontrail.org/29093
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-02-24: [Review update] R3.0

Review in progress for https://review.opencontrail.org/29094
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-02-28: A change has been merged

#13

Reviewed: https://review.opencontrail.org/29094
Committed: http://github.org/Juniper/contrail-controller/commit/e046ce9e01f0769014096202fbf62f6aa8c82ecc
Submitter: Zuul (<email address hidden>)
Branch: R3.0

commit e046ce9e01f0769014096202fbf62f6aa8c82ecc
Author: ashoksingh <email address hidden>
Date: Thu Feb 23 17:49:27 2017 +0530

Fix issue of VM not getting deleted from Agent

Closes-Bug: #1665309
(cherry picked from commit e522093fdf6c18260a7e742d05f96b2e601ce3b5)

Change-Id: I182465602b64b93e83c4904efe086d054acd9f16

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-02-28:

#14

Reviewed: https://review.opencontrail.org/29093
Committed: http://github.org/Juniper/contrail-controller/commit/22e1169a63e48b0c66f13cae2e40c6618f44b0b3
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit 22e1169a63e48b0c66f13cae2e40c6618f44b0b3
Author: ashoksingh <email address hidden>
Date: Thu Feb 23 17:49:27 2017 +0530

Fix issue of VM not getting deleted from Agent

Closes-Bug: #1665309
(cherry picked from commit e522093fdf6c18260a7e742d05f96b2e601ce3b5)

Change-Id: Ibab72ee82c55c7b4225c68e3d852cdebe2dde511

Revision history for this message

Ashok Singh (ashoksr) wrote on 2017-03-01:

#15

Issue is specific to Juno release. R3.2 and mainline don't have Juno releases

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-03-01: [Review update] R3.0.2.x

#16

Review in progress for https://review.opencontrail.org/29205
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-03-17: A change has been merged

#18

Reviewed: https://review.opencontrail.org/29205
Committed: http://github.org/Juniper/contrail-controller/commit/adc24f2fbd0e41364ed9ac8a5631bc5cec017042
Submitter: Zuul (<email address hidden>)
Branch: R3.0.2.x

commit adc24f2fbd0e41364ed9ac8a5631bc5cec017042
Author: ashoksingh <email address hidden>
Date: Wed Mar 1 14:12:01 2017 +0530

Fix issue of VM not getting deleted from Agent

Nova repo is absent in R3.0.2.x/centos71/manifest-juno.xml. This repo has changes to invoke/use Agent’s REST API for port Adds/deletes.
Until we have this repo in all manifest files of R3.0.2.x branches continue to use Thrift instead of REST.

This reverts the commit https://github.com/Juniper/contrail-controller/commit/bd890daadb06bb8341bfdf217ab2c0c53988311d

Change-Id: Id30bc9d3a5eeab7b2f459a9073e3d56fe0fe9168
Closes-Bug: #1665309