[Analytics] Make some of the contrail analytics components installation optional

Bug #1784493 reported by Biswajit Mandal
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
In Progress
High
Biswajit Mandal
Trunk
Fix Committed
High
Biswajit Mandal

Bug Description

Make kafka/alarm-gen/snmp-collector/topology installation optional.

Jeba Paulaiyan (jebap)
tags: added: fabric
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R5.0

Review in progress for https://review.opencontrail.org/45627
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45970
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46218
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46219
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46220
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46223
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

information type: Proprietary → Public
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46223
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/45827
Committed: http://github.com/Juniper/contrail-analytics/commit/906030b2e1a4be6fd34aef2764fc4169154ed5b8
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 906030b2e1a4be6fd34aef2764fc4169154ed5b8
Author: Biswajit Mandal <email address hidden>
Date: Tue Aug 28 20:56:20 2018 +0000

Blueprint for making some analytics components installation optional.
Will be adding below 3 new roles:
analytics_snmp - contains snmp/topology components
analytics_alarm - contains alarm/kafka components
analytics_database - contains query-engine/cassandra components

zookeeper is removed from analytics_database and all the components use zookeeper from config_database

Blueprint Link:
==============

https://blueprints.launchpad.net/opencontrail/+spec/optional-components-in-contrail-analytics

Change-Id: I24ab43392c78823e2fc9cc4e30f422866a4b3a2e
Partial-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/46700
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46699
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46700
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46699
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46700
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46929
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46218
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46700
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/46700
Committed: http://github.com/Juniper/contrail-analytics/commit/4988692fc4aed08123353308018401ac56ecfbfa
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 4988692fc4aed08123353308018401ac56ecfbfa
Author: Biswajit Mandal <email address hidden>
Date: Thu Oct 4 00:38:47 2018 +0000

If local redis to Analytics-api is down, then query-engine stops working.

Queries sent from Analytics is reached to Query engine by local redis. Query engine listens to local redis to get the query.
Now change is:
Query engine should listen to all redis as specified in config REDIS.server_list which is list of ip:port format.

If any one is down, then also it should be able to process the request as posted to different
redis.

Manual testing done in a 3 node (node1, node2, node3) are as below:
1. Initially all QE are up, and they are using all redis for Redis-Query and in server_addrs, shows all redis instances
2. Make redis down in node1, make sure NodeStatus of QE for Redis-Query shows UP, and in description shows node1 redis as down.
3. Make node2 redis also down, make sure NodeStatus of QE for Redis-Query shows UP, and in description shows node1 & node2 redis as down.
4. Make redis in node3 also down, then check Redis-Query shows as down.
5. Now make any one redis instance UP, make sure check Redis-Query connection Status becomes UP, and description shows that other two connection as down
6. Now make other two nodes' redis also UP, verify Redis-Query connection Status becomes UP, and description becomes null in NodeStatus.
In the above all cases, server_addrs shows all redis instances, and description shows which redis is down.

Change-Id: Iff7970d6f82f869d8c48ed1abfa1250fdba87c6a
Related-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/46699
Committed: http://github.com/Juniper/contrail-container-builder/commit/dc778f5085cb5c701f03e7e2e8c7793065dae240
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit dc778f5085cb5c701f03e7e2e8c7793065dae240
Author: Biswajit Mandal <email address hidden>
Date: Sat Oct 6 01:25:47 2018 +0000

Adding REDIS.server_list to have all the redis list which is list of redis nodes with IP:Port format

Will remove REDIS.server and REDIS.port configuration in next check-in.

Related-Bug: #1784493
Depends-On: Iff7970d6f82f869d8c48ed1abfa1250fdba87c6a

Change-Id: I360af3b8641764c354761332fa2b64a62fb0403c

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/47720
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/47720
Committed: http://github.com/Juniper/contrail-analytics/commit/893f2486c1e9826b2ea3e8d3afeb5d5adcbf3d82
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 893f2486c1e9826b2ea3e8d3afeb5d5adcbf3d82
Author: Biswajit Mandal <email address hidden>
Date: Fri Nov 16 14:50:28 2018 -0800

Corrected the blueprint format

Change-Id: I74e3171e29fd7e8a09200d428a6189c68c0f8bec
Related-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/46597
Committed: http://github.com/Juniper/contrail-container-builder/commit/4f47c32887605b703a71bc549d07da7e1b203996
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 4f47c32887605b703a71bc549d07da7e1b203996
Author: Biswajit Mandal <email address hidden>
Date: Fri Sep 28 19:18:06 2018 +0000

Based on the number of alarmgen installed, each will have redis_agg_db offset set in environment
variable, alarmgen will read it and use it for uve aggregation DB selection (redis_agg_db offset + 6)
redis_agg_db offset starts from 1

now offset is calcualted based on number of analytics node is installed, once Bug 1784493 is merged,
then we need to use ANALYTICS_ALARM

Change-Id: I1b1228a026dd0cdc6e65cb13bb9f9dcbf33b6b35
Related-Bug: #1793847

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46218
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46929
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46218
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/46220
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/45083
Committed: http://github.com/Juniper/contrail-ansible-deployer/commit/a2ef99bda28a6e9a858d4fce2e3678307a5f54f7
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit a2ef99bda28a6e9a858d4fce2e3678307a5f54f7
Author: Biswajit Mandal <email address hidden>
Date: Fri Dec 21 00:37:33 2018 +0000

Making kafka/alarm-gen/snmp-collector/topology/zookeeper/cassandra/query-engine installation optional.

Three new roles are introduced.

analytics_alarm - If added, alarm-gen, kafka will be installed
analytics_snmp - If added, snmp-collector and topology will be installed

If we need to enable analytics_alarm role, then analytics_database_kafka role also needs to
be added in any of the deploying node in cluster.

All components will use config_database zookeeper except kafka.

Partial-Bug: #1784493
Depends-On: I8eb42992351595c086b6b2e3eeac0ff9253de599
Depends-On: I8859139f200cbe57c48031d495b877d7def1f992
Depends-On: Icbf07f470989f7a70f14d6bb62e6c9fbd7194ebb
Depends-On: Idee370fc9ed1b42320611eba88d79cc6054d2c5f
Depends-On: I4758f190fb9ad0238774965da8b74f0bdb5b4943
Depends-On: I2ae23faa5ba72408018751012e31374ef51e16c4

Change-Id: I45ab1a28733d9b445ca50bac38a5502e6c3c5f26

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/45643
Committed: http://github.com/Juniper/contrail-container-builder/commit/b3d7835e47be02c3a7d518049b74b32ab5550ea9
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit b3d7835e47be02c3a7d518049b74b32ab5550ea9
Author: Biswajit Mandal <email address hidden>
Date: Thu Jan 10 10:13:14 2019 +0000

Do not populate kafka related config in collector config file if ENABLE_ANALYTICS_DATABASE_KAFKA is set as false or undefined.
Call introspect call for contrail-status for kafka if ENABLE_ANALYTICS_DATABASE_KAFKA is set as True
snmp-collector/topology if ENABLE_ANALYTICS_SNMP is set as True, alarm if ENABLE_ANALYTICS_ALARM is set to true

Introduced new variable for cassandra/kafka db based on new roles.

Config File changes:

alarm-gen:
{{zk_list}}: zookeeper list from configdb

api:
{{partitions}}: Only if Kafka enabled
{{zk_list}}: zookeeper list from configdb
{{redis_query_port}}: Only if Cassandra enabled

collector:
{{kafka_broker_list}}: Only if Kafka enabled
{{zookeeper_server_list}}: zookeeper list from configdb
{{[DATABASE]}}: Only of Cassandra enabled

query-engine:
No Change

snmp-collector:
{{zookeeper}}: zookeeper list from configdb
topology:
{{zookeeper}}: zookeeper list from configdb

Conflicts:
containers/external/kafka/docker-entrypoint.sh

Change-Id: I8eb42992351595c086b6b2e3eeac0ff9253de599
Partial-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :
Download full text (4.1 KiB)

Reviewed: https://review.opencontrail.org/45972
Committed: http://github.com/Juniper/contrail-analytics/commit/66e1d811b97ccf9e534dff3312a601ac9d2e6cf2
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 66e1d811b97ccf9e534dff3312a601ac9d2e6cf2
Author: Biswajit Mandal <email address hidden>
Date: Thu Dec 20 19:25:14 2018 +0000

The below REST GET APIs should not be visible in api if analytics_alarm role is not provisioned.
/analytics/alarms
/analytics/alarm-stream
/analytics/uve-stream

If analytics_database is not provisioned, then the below REST APIs should not be visible in api

GET API
/analytics/tables
/analytics/queries

POST API
/analytics/query

The UT result with all changes in the sandbox locally as below:

test_10_multiple_or_query OK 0.01
test_1_no_arg OK 0.01
test_2_message_query OK 0.01
test_3_object_value OK 0.01
test_4_object_id OK 0.01
test_5_prefix_query OK 0.01
test_6_long_query OK 0.01
test_7_multiple_sources_query OK 0.01
test_8_multiple_modules_query OK 0.01
test_9_multiple_message_types_query OK 0.01
test.test_overlay_to_underlay_mapper.TestOverlayToUnderlayMapper
test_get_overlay_flow_data_noerror OK 0.00
test_get_overlay_flow_data_raise_exception OK 0.00
test_get_underlay_flow_data_noerror OK 0.00
test_get_underlay_flow_data_raise_exception OK 0.00
test_process_query OK 0.00
test_send_query_no_error OK 0.00
test_send_query_raise_exception OK 0.00
test_send_response_no_error OK 0.00
test_send_response_raise_exception OK 0.00
test.test_flow.FlowQuerierTest
test_1_noarg_query OK 0.00
test.test_stats.StatQuerierTest
test_1_analytics_cpu_query OK 0.01
test.test_uveserver.UVEServerTest
test_append_agg OK 0.00
test_default_agg OK 0.00
test_elem_sum_agg OK 0.00
test_list_union_agg OK 0.00
test_map_union_agg OK 0.00
test_simple OK 0.00
test_struct_sum_agg OK 0.00
test.test_sessions.StatQuerierTest
test_0_session_series_query OK 0.00
test_1_session_record_query OK 0....

Read more...

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/46218
Committed: http://github.com/Juniper/contrail-controller/commit/53eafb7a9227fdea8a24eb569b428850e8f5af4e
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 53eafb7a9227fdea8a24eb569b428850e8f5af4e
Author: Biswajit Mandal <email address hidden>
Date: Wed Dec 19 20:35:42 2018 +0000

Add analytics-alarm and alalytics-snmp nodemgr.
Adding provisiong code for alarm/snmp nodes

Partial-Bug: #1784493

Change-Id: Icbf07f470989f7a70f14d6bb62e6c9fbd7194ebb

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/46220
Committed: http://github.com/Juniper/contrail-api-client/commit/48634a527ab8217aea581c23437b69ef89431d02
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 48634a527ab8217aea581c23437b69ef89431d02
Author: Biswajit Mandal <email address hidden>
Date: Mon Sep 10 18:23:26 2018 +0000

Adding schema for analytics-snmp/alarm node types
Partial-Bug: #1784493

Conflicts:

 schema/vnc_cfg.xsd
Change-Id: I4758f190fb9ad0238774965da8b74f0bdb5b4943

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/46223
Committed: http://github.com/Juniper/contrail-zuul-jobs/commit/4e1b80a3de96943af7caf6492a30f2043416ae90
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 4e1b80a3de96943af7caf6492a30f2043416ae90
Author: Biswajit Mandal <email address hidden>
Date: Sun Sep 16 05:19:00 2018 +0000

Adding analytics_snmp/_alarm roles with zuul testing for CI through ansible-deployer.

Change-Id: Idee370fc9ed1b42320611eba88d79cc6054d2c5f
Partial-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/46929
Committed: http://github.com/Juniper/contrail-test/commit/db66a985401b916c85838ac6965ea6c8fbdf73b4
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit db66a985401b916c85838ac6965ea6c8fbdf73b4
Author: Biswajit Mandal <email address hidden>
Date: Thu Oct 11 23:11:17 2018 +0000

With changes of optional components, moving query-engine as part of analytics_database, removing zookeeper from analyticsdb

Related-Bug: #1784493

Conflicts:
 common/contrail_services.py
Change-Id: I2ae23faa5ba72408018751012e31374ef51e16c4

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] stable/queens

Review in progress for https://review.opencontrail.org/49120
Submitter: alexey-mr (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/49120
Committed: http://github.com/Juniper/contrail-tripleo-heat-templates/commit/e6702a1e5ca07c994a6888fdf33093f7c1281679
Submitter: Zuul v3 CI (<email address hidden>)
Branch: stable/queens

commit e6702a1e5ca07c994a6888fdf33093f7c1281679
Author: alexey-mr <email address hidden>
Date: Mon Feb 4 15:17:30 2019 +0300

Separate service for snmp and alarm

- it is to reflect new structure in contrail
containers and makes an ability to have alarm
and snmp optional

Change-Id: Iea9eda203058f3ee433d8cc81a81e0f61bb498f9
Partial-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/49454
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/49454
Committed: http://github.com/Juniper/contrail-ansible-deployer/commit/f16cb356e7f4be0074a30bf54b1efb40bfb37fa1
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit f16cb356e7f4be0074a30bf54b1efb40bfb37fa1
Author: Biswajit Mandal <email address hidden>
Date: Sat Feb 16 05:26:01 2019 +0000

Fixing the alarm-nodes for analytics_snmp role.

Change-Id: I6e0737d4a6883788bd920353dd9036d45dc37df6
Closes-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/49535
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged
Download full text (8.3 KiB)

Reviewed: https://review.opencontrail.org/49387
Committed: http://github.com/Juniper/contrail-container-builder/commit/0abd877f54b448e8bef01f2d817d5bf1c97649c4
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: R6.0-WIP

commit 0abd877f54b448e8bef01f2d817d5bf1c97649c4
Author: Biswajit Mandal <email address hidden>
Date: Fri Sep 28 19:18:06 2018 +0000

Apply commits from master onto R6.0-WIP

Based on the number of alarmgen installed, each will have redis_agg_db offset set in environment
variable, alarmgen will read it and use it for uve aggregation DB selection (redis_agg_db offset + 6)
redis_agg_db offset starts from 1
now offset is calcualted based on number of analytics node is installed, once Bug 1784493 is merged,
then we need to use ANALYTICS_ALARM
Related-Bug: #1793847

copy job-manager dependencies to device-manager container
next review will remove it from config-api
Partial-Bug: #1794549

nodemgr: install chrony and ntp to be able to check time sync everywhere
Closes-Bug: #1800704

Add STAT collection destination to the daemon config files
The STAT collector socket path specified in the instances.yaml
have to be populated in the daemon config files.
Closes-Bug: 1794107

Updated mellanox plugin repositories
Partial-bug: #1765858
Signed-off-by: Vladimir Sokolovsky <email address hidden>

Add specific IP option for Introspect in contrail-vcenter-manager config file
Partial-Bug: #1802626

Updated mellanox plugin redhat version to 7.6
Partial-bug: #1765858
Signed-off-by: Matan Azrad <email address hidden>

implement SSL support rabbitmq container
partial-jira-bug: JCB-218684

add container's ID to contrail-status output
closes-jira-bug: JCB-218706

Fixed contrail-status container opts for IPA case
In case of IPA ca file is mounted separately and placed
in /etc/ipa/ca.crt. Node init should skip cert generation,
so is_ssl_enabled is false for it, so, ssl folder is to always mounted.
In case of no ssl it is just empty.
Partial-Bug: #1805443

remove copy-paste from external containers
external containers didn't have access to common definitions and functions.
solution is to copy common.sh and functions into external containers and use
code from it to reduce copy-paste.
closes-jira-bug: JCB-218707

remove call to pre-start-init from externals for now
it waits for certs if SSL_ENABLE is true. but not all containers
have /etc/contrail/ssl mount and some has own flag for SSL.
closes-jira-bug: JCB-218707

implement node-to-node encryption for cassandra
partial-jira-bug: JCB-218753

remove obsolete code and move some code from general-base to specific containers
closes-jira-bug: JCB-218707

use common function in rabbitmq's entrypoint
now rabbitmq has access to common functions and should us them
to avoid copy-paste.
closes-jira-bug: JCB-218707

Added cacert parametet for contrail-status
- in case of IPA cacert is different from
default value
Partial-Bug: #1805443

install pip via easy_install
installing python-pip in RHEL via yum is very ugly.
And anyway after installing pip code upgrades it from pypi.
partial-jira-bug: JCB-218785

create dnsmasq container and remove tftp/dhcp
cleanup obsolete dependencies from config-api (as they moved...

Read more...

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :
Download full text (5.3 KiB)

Reviewed: https://review.opencontrail.org/49385
Committed: http://github.com/Juniper/contrail-ansible-deployer/commit/6d56f894b1547baddfc84059e0f6ab3cd34dec8e
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: R6.0-WIP

commit 6d56f894b1547baddfc84059e0f6ab3cd34dec8e
Author: Michael Henkel <email address hidden>
Date: Mon Dec 3 02:49:13 2018 +0100

Apply commits from master onto R6.0-WIP

ansible-deployer use correct InternalIP for kubelet
Closes-Bug: 1806299

Use agent's entrypoint on Windows
Start using agent's entrypoint to achieve similar behaviour
to Linux's microservices.
Partial-Bug: #1806677

Starting docker service before pull
Closes-Bug: #1807716

Closes-Bug: #1808241 - Protect against reprovision failures
VROUTER_NODES_LIST is not needed - causes reprovision un-necessarily
Also, Openstack might be in half-configured state if provision fails first time due to intermittent timeout
Here re-provision shouldn't fail due to missing endpoints from first run.

Support stats collector destination in ansible
Stat messages are written to socket file which will
be read by stats collector. The socket file to be
written to is specified in instances.yaml. The folder
holfing the socket file will be mounted by all the
containers. The user specified STATS_COLLECTOR_DESTINATION_PATH
in the instances.yaml.
Partial-Bug: 1794107

add ssl mounts for rabbitmq and cassandra
closes-jira-bug: JCB-218684

change configuration description for additional rabbitmq options
partial-jira-bug: JCB-218684

Handle domainsuffix for Windows nodes
1. Expose provider_config to install container roles
2. Use proper FQDN in POST to config node
Closes-Bug: #1810173

Add domainsuffix to Windows examples
Closes-Bug: #1810173

Change default openstack version from ocata to queens
Closes Jira Bug: JCB-218736

Use kolla playbook to generate self-signed certs
If generate_self_signed_certs is set to yes under kolla_config->kolla_globals
section of the instance.yml file, then generate self-signed certs to be used for
the openstack haproxy service. Note that this will be used only when
kolla_enable_tls_external is set to true.
If using already generated certs, then give the path to the certificates using
the kolla_external_fqdn_cert variable
Partial Jira Bug: JCB-218738

Making kafka/alarm-gen/snmp-collector/topology/zookeeper/cassandra/query-engine installation optional.
Three new roles are introduced.
analytics_alarm - If added, alarm-gen, kafka will be installed
analytics_snmp - If added, snmp-collector and topology will be installed
If we need to enable analytics_alarm role, then analytics_database_kafka role also needs to
be added in any of the deploying node in cluster.
All components will use config_database zookeeper except kafka.
Partial-Bug: #1784493

add dnsmasq to config-api pod
next commits will move device manager and dnsmasq to separate pod
partial-jira-bug: JCB-218801

add capabilities required for dnsmasq
partial-jira-bug: JCB-218801

Fix extracting hostname from Windows
Use `hostname` bash command instead of ansible_hostname
Closes Jira Bug: TFB-1370

Initial commit for Openstack Rocky support
Partial-Jira-Bug: JCB-218848

fix issues preventing to use...

Read more...

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/49583
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/49583
Committed: http://github.com/Juniper/contrail-ansible-deployer/commit/461eb5e615adab6c2aee5ed556f0c8bec5514904
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 461eb5e615adab6c2aee5ed556f0c8bec5514904
Author: Biswajit Mandal <email address hidden>
Date: Wed Feb 20 23:31:29 2019 +0000

Updating the README.md file as we removed zookeeper from analytics.

Change-Id: I27e51a9822b6b76376efe16c0de9a887002080a7
Closes-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/49639
Submitter: Biswajit Mandal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/49639
Committed: http://github.com/Juniper/contrail-controller/commit/2feae44dcdc46cb97eef11f1765bad626ce8e6b9
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 2feae44dcdc46cb97eef11f1765bad626ce8e6b9
Author: Biswajit Mandal <email address hidden>
Date: Thu Feb 21 20:26:06 2019 +0000

ContrailConfig missing for alarm/snmp node and alarm not raised for components tied in analytics_alarm
and analytics_snmp role

Fixes two issues:
1. Added two roles analytics_alarm and analytics_snmp in _UVEMAP while creating VncDbClient object
2. Added the new roles inside uve_key of alarm_list.

Change-Id: Ia14c8d3b131185e751aeb8189df9bf2b3712e9cf
closes-jira-bug: JCB-219332
Closes-Bug: #1784493

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/49535
Committed: http://github.com/Juniper/contrail-controller/commit/17c0be3002f723bd589af526f0d9606df0ce7498
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 17c0be3002f723bd589af526f0d9606df0ce7498
Author: Biswajit Mandal <email address hidden>
Date: Tue Feb 19 23:30:23 2019 +0000

While importing cassandra & zookeeper DB through db_json_exim.py, the metadata info is skipped in zookeeper.
So adding /analytics-discovery- also in ignore list.

Change-Id: I84d4cd581b4d696c180e18901da4d78f716f9c96
Closes-Bug: #1784493

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.