Juniper Openstack

[3058-mainline] All deleted VNs are showing in Analytics UVE

Bug #1686668 reported by musharani on 2017-04-27

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
Juniper Openstack	Status tracked in Trunk
R4.0	Fix Committed	Medium	Jack Jonnalagadda	Juniper Openstack r4.0.2.0
R4.1	Fix Committed	Medium	Jack Jonnalagadda	Juniper Openstack r4.1.0.0-fcs "r4.1"
Trunk	Fix Committed	Medium	Jack Jonnalagadda	Juniper Openstack r5.0.0

Bug Description

In Analytics UVE all the deleted networks are showing. But the same thing got removed from config-api server.

From config api server:
------------------------
{
virtual-networks: [
{
href: "http://nodec59.englab.juniper.net:8080/proxy?proxyURL=http://10.204.217.99:8082/virtual-network/187f0db3-49f9-43f0-a2f9-1f622a1ebea8",
fq_name: [
"default-domain",
"default-project",
"ip-fabric"
],
uuid: "187f0db3-49f9-43f0-a2f9-1f622a1ebea8"
},
{
href: "http://nodec59.englab.juniper.net:8080/proxy?proxyURL=http://10.204.217.99:8082/virtual-network/295b182d-270a-4489-82a2-54bc65280300",
fq_name: [
"default-domain",
"default-project",
"default-virtual-network"
],
uuid: "295b182d-270a-4489-82a2-54bc65280300"
},
{
href: "http://nodec59.englab.juniper.net:8080/proxy?proxyURL=http://10.204.217.99:8082/virtual-network/d8a0bcb3-be49-4ddd-a6eb-43d39a2c7b47",
fq_name: [
"default-domain",
"default-project",
"__link_local__"
],
uuid: "d8a0bcb3-be49-4ddd-a6eb-43d39a2c7b47"
}
]
}

But if you check the analytics uve it is showing more than 40 vns. All those vns are deleted from UI.
http://nodec59.englab.juniper.net:8080/proxy?proxyURL=http://10.204.217.99:8081/analytics/uves/virtual-networks

Setup is in the same state. You can use the setup to debug if you need.

Setup details:
--------------
nodec59 - root/c0ntrail123

Build:
------
3058 -ubuntu-mitaka

Tags:

Revision history for this message

musharani (musharani) wrote on 2017-04-27:

analytics_uve.png Edit (161.5 KiB, image/png)

Revision history for this message

musharani (musharani) wrote on 2017-04-27:

config-api.png Edit (108.9 KiB, image/png)

Revision history for this message

musharani (musharani) wrote on 2017-04-27:

monitor-networks.png Edit (131.7 KiB, image/png)

summary:

- [3058-mainline] Deleted all the VNs are showing in Analytics UVE
+ [3058-mainline] All deleted VNs are showing in Analytics UVE

Revision history for this message

Raj Reddy (rajreddy) wrote on 2017-04-27:

This is most likely related to
https://bugs.launchpad.net/juniperopenstack/+bug/1685334

we will verify once that's fixed and 'dup' this.

Revision history for this message

musharani (musharani) wrote on 2017-08-10:

Still the issue is seen. So it is not related to the bug https://bugs.launchpad.net/juniperopenstack/+bug/1685334. So remove the dup bug link

Revision history for this message

Sundaresan Rajangam (srajanga) wrote on 2017-09-06:

This seem to be same as https://bugs.launchpad.net/juniperopenstack/+bug/1709161
Can you please verify this in the latest build? If you see this issue in the latest build, please provide the setup.

Revision history for this message

Sundaresan Rajangam (srajanga) wrote on 2017-09-08:

When you see the issue next time, can you get the output of /analytics/uves/virtual-network/*?flat

Revision history for this message

musharani (musharani) wrote on 2017-09-22:

analytics_uve_vn.png Edit (129.3 KiB, image/png)

Still the issue is seen in the 4.0 - 31

output of /analytics/uves/virtual-network/*?flat/:
{
value: [ ]
}

Please find the attached uve link
https://nodec41.englab.juniper.net:8143/proxy?proxyURL=http://10.204.217.81:8081/analytics/uves/virtual-networks

vn-test network got deleted from config page. But still it is showing on analytics.

Analytics is having all the deleted vns from different projects which are all deleted. Setup is in failed state. You can use the setup nodec41.

Revision history for this message

musharani (musharani) wrote on 2017-09-22:

analytics_uve_vn.png Edit (129.3 KiB, image/png)

Still the issue is seen in the 4.0 - 31

output of /analytics/uves/virtual-network/*?flat/:
{
value: [ ]
}

Please find the attached uve link
https://nodec41.englab.juniper.net:8143/proxy?proxyURL=http://10.204.217.81:8081/analytics/uves/virtual-networks

vn-test network got deleted from config page. But still it is showing on analytics.

Analytics is having all the deleted vns from different projects which are all deleted. Setup is in failed state. You can use the setup nodec41.

Revision history for this message

Sundaresan Rajangam (srajanga) wrote on 2017-09-22:

#10

Download full text (6.3 KiB)

If the virtual-network is deleted, the derived stats of the corresponding virtual-network doesn't get deleted.

root@nodec41(analytics):/# curl -u admin:contrail123 localhost:8181/analytics/uves/virtual-network/default-domain:ctest-WebuiTestSanity-08644118:left_vn?flat | python -m json.tool
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 1218 0 1218 0 0 75333 0 --:--:-- --:--:-- --:--:-- 76125
{
    "AggProxySum-UveVirtualNetworkAgent-egress_flow_count": {
        "deleted": false,
        "proxy": "UveVirtualNetworkAgent-egress_flow_count",
        "value": {
            "value": 0
        }
    },
    "AggProxySum-UveVirtualNetworkAgent-ingress_flow_count": {
        "deleted": false,
        "proxy": "UveVirtualNetworkAgent-ingress_flow_count",
        "value": {
            "value": 0
        }
    },
    "RoutingInstanceStatsData": {
        "ermvpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "evpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv4_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv6_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        }
    }
}

If the virtual-network is deleted, the derived stats of the corresponding virtual-network doesn't get deleted.

root@nodec41(analytics):/# curl -u admin:contrail123 localhost:8181/analytics/uves/virtual-network/default-domain:ctest-WebuiTestSanity-08644118:left_vn?flat | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1218    0  1218    0     0  75333      0 --:--:-- --:--:-- --:--:-- 76125
{
    "AggProxySum-UveVirtualNetworkAgent-egress_flow_count": {
        "deleted": false,
        "proxy": "UveVirtualNetworkAgent-egress_flow_count",
        "value": {
            "value": 0
        }
    },
    "AggProxySum-UveVirtualNetworkAgent-ingress_flow_count": {
        "deleted": false,
        "proxy": "UveVirtualNetworkAgent-ingress_flow_count",
        "value": {
            "value": 0
        }
    },
    "RoutingInstanceStatsData": {
        "ermvpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "evpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv4_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv6_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:left_vn:left_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        }
    }
}

root@nodec41(analytics):/# curl -u admin:contrail123 localhost:8181/analytics/uves/virtual-network/default-domain:ctest-WebuiTestSanity-84112158:vn-test?flat | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   912    0   912    0     0   127k      0 --:--:-- --:--:-- --:--:--  148k
{
    "RoutingInstanceStatsData": {
        "ermvpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-84112158:vn-test:vn-test\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "evpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-84112158:vn-test:vn-test\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv4_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-84112158:vn-test:vn-test\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv6_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-84112158:vn-test:vn-test\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        }
    }
}

root@nodec41(analytics):/# curl -u admin:contrail123 localhost:8181/analytics/uves/virtual-network/default-domain:ctest-WebuiTestSanity-08644118:right_vn?flat | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1226    0  1226    0     0  75279      0 --:--:-- --:--:-- --:--:-- 76625
{
    "AggProxySum-UveVirtualNetworkAgent-egress_flow_count": {
        "deleted": false,
        "proxy": "UveVirtualNetworkAgent-egress_flow_count",
        "value": {
            "value": 0
        }
    },
    "AggProxySum-UveVirtualNetworkAgent-ingress_flow_count": {
        "deleted": false,
        "proxy": "UveVirtualNetworkAgent-ingress_flow_count",
        "value": {
            "value": 0
        }
    },
    "RoutingInstanceStatsData": {
        "ermvpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:right_vn:right_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "evpn_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:right_vn:right_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv4_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:right_vn:right_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        },
        "ipv6_stats": {
            "nodec41:Control:contrail-control:0:\"default-domain:ctest-WebuiTestSanity-08644118:right_vn:right_vn\"": {
                "infeasible_paths": 0,
                "prefixes": 0,
                "primary_paths": 0,
                "secondary_paths": 0,
                "total_paths": 0
            }
        }
    }
}

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-09-29: [Review update] master

#11

Review in progress for https://review.opencontrail.org/36115
Submitter: Jack Jonnalagadda (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-10-04: [Review update] R4.0

#13

Review in progress for https://review.opencontrail.org/36207
Submitter: Jack Jonnalagadda (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-10-04: A change has been merged

#15

Reviewed: https://review.opencontrail.org/36115
Committed: http://github.com/Juniper/contrail-sandesh/commit/91e89a8f5cf04e6ff077532a63da0d7a8712d6d8
Submitter: Zuul (<email address hidden>)
Branch: master

commit 91e89a8f5cf04e6ff077532a63da0d7a8712d6d8
Author: VS Jagannadham Jonnalagadda <email address hidden>
Date: Thu Sep 28 18:30:45 2017 -0700

Delete UVEs that are timedout
Closes-Bug: 1686668

Change-Id: I1e2c304d5eacb02aed068ca4bf80936ebccb8288

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-10-04:

#16

Reviewed: https://review.opencontrail.org/36207
Committed: http://github.com/Juniper/contrail-sandesh/commit/d1d9c9f58ee9e2538dd3545b691e613eff9aaf92
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit d1d9c9f58ee9e2538dd3545b691e613eff9aaf92
Author: VS Jagannadham Jonnalagadda <email address hidden>
Date: Thu Sep 28 18:30:45 2017 -0700

Delete UVEs that are timedout
Closes-Bug: 1686668

Change-Id: I1e2c304d5eacb02aed068ca4bf80936ebccb8288

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-10-27: [Review update] R4.1

#17

Review in progress for https://review.opencontrail.org/36939
Submitter: Jack Jonnalagadda (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2017-11-02: A change has been merged

#19

Reviewed: https://review.opencontrail.org/36939
Committed: http://github.com/Juniper/contrail-sandesh/commit/a428d456ada67805a2f1061e6c16e875fefea7a5
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit a428d456ada67805a2f1061e6c16e875fefea7a5
Author: VS Jagannadham Jonnalagadda <email address hidden>
Date: Thu Sep 28 18:30:45 2017 -0700

Delete UVEs that are timedout
Closes-Bug: 1686668

Change-Id: I1e2c304d5eacb02aed068ca4bf80936ebccb8288
(cherry picked from commit 91e89a8f5cf04e6ff077532a63da0d7a8712d6d8)

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Bug attachments

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.