[Analytics] Make some of the contrail analytics components installation optional

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45627
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45970
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/46218
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45972
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/46219
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/46220
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/46223
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45643
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45083
Submitter: Biswajit Mandal (<email address hidden>)

Review in progress for https://review.opencontrail.org/45827
Submitter: Biswajit Mandal (<email address hidden>)

Reviewed: https://review.opencontrail.org/46218
Committed: http://github.com/Juniper/contrail-controller/commit/53eafb7a9227fdea8a24eb569b428850e8f5af4e
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 53eafb7a9227fdea8a24eb569b428850e8f5af4e
Author: Biswajit Mandal <email address hidden>
Date: Wed Dec 19 20:35:42 2018 +0000

Add analytics-alarm and alalytics-snmp nodemgr.
Adding provisiong code for alarm/snmp nodes

Partial-Bug: #1784493

Change-Id: Icbf07f470989f7a70f14d6bb62e6c9fbd7194ebb

Reviewed: https://review.opencontrail.org/46220
Committed: http://github.com/Juniper/contrail-api-client/commit/48634a527ab8217aea581c23437b69ef89431d02
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 48634a527ab8217aea581c23437b69ef89431d02
Author: Biswajit Mandal <email address hidden>
Date: Mon Sep 10 18:23:26 2018 +0000

Adding schema for analytics-snmp/alarm node types
Partial-Bug: #1784493

Conflicts:

 schema/vnc_cfg.xsd
Change-Id: I4758f190fb9ad0238774965da8b74f0bdb5b4943

Reviewed: https://review.opencontrail.org/46223
Committed: http://github.com/Juniper/contrail-zuul-jobs/commit/4e1b80a3de96943af7caf6492a30f2043416ae90
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit 4e1b80a3de96943af7caf6492a30f2043416ae90
Author: Biswajit Mandal <email address hidden>
Date: Sun Sep 16 05:19:00 2018 +0000

Adding analytics_snmp/_alarm roles with zuul testing for CI through ansible-deployer.

Change-Id: Idee370fc9ed1b42320611eba88d79cc6054d2c5f
Partial-Bug: #1784493

Reviewed: https://review.opencontrail.org/46929
Committed: http://github.com/Juniper/contrail-test/commit/db66a985401b916c85838ac6965ea6c8fbdf73b4
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: master

commit db66a985401b916c85838ac6965ea6c8fbdf73b4
Author: Biswajit Mandal <email address hidden>
Date: Thu Oct 11 23:11:17 2018 +0000

With changes of optional components, moving query-engine as part of analytics_database, removing zookeeper from analyticsdb

Related-Bug: #1784493

Conflicts:
 common/contrail_services.py
Change-Id: I2ae23faa5ba72408018751012e31374ef51e16c4

Review in progress for https://review.opencontrail.org/49120
Submitter: alexey-mr (<email address hidden>)

Reviewed: https://review.opencontrail.org/49120
Committed: http://github.com/Juniper/contrail-tripleo-heat-templates/commit/e6702a1e5ca07c994a6888fdf33093f7c1281679
Submitter: Zuul v3 CI (<email address hidden>)
Branch: stable/queens

commit e6702a1e5ca07c994a6888fdf33093f7c1281679
Author: alexey-mr <email address hidden>
Date: Mon Feb 4 15:17:30 2019 +0300

Separate service for snmp and alarm

- it is to reflect new structure in contrail
containers and makes an ability to have alarm
and snmp optional

Change-Id: Iea9eda203058f3ee433d8cc81a81e0f61bb498f9
Partial-Bug: #1784493

Review in progress for https://review.opencontrail.org/49454
Submitter: Biswajit Mandal (<email address hidden>)

Reviewed: https://review.opencontrail.org/49454
Committed: http://github.com/Juniper/contrail-ansible-deployer/commit/f16cb356e7f4be0074a30bf54b1efb40bfb37fa1
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit f16cb356e7f4be0074a30bf54b1efb40bfb37fa1
Author: Biswajit Mandal <email address hidden>
Date: Sat Feb 16 05:26:01 2019 +0000

Fixing the alarm-nodes for analytics_snmp role.

Change-Id: I6e0737d4a6883788bd920353dd9036d45dc37df6
Closes-Bug: #1784493

Review in progress for https://review.opencontrail.org/49535
Submitter: Biswajit Mandal (<email address hidden>)

Reviewed: https://review.opencontrail.org/49387
Committed: http://github.com/Juniper/contrail-container-builder/commit/0abd877f54b448e8bef01f2d817d5bf1c97649c4
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: R6.0-WIP

commit 0abd877f54b448e8bef01f2d817d5bf1c97649c4
Author: Biswajit Mandal <email address hidden>
Date: Fri Sep 28 19:18:06 2018 +0000

Apply commits from master onto R6.0-WIP

Based on the number of alarmgen installed, each will have redis_agg_db offset set in environment
variable, alarmgen will read it and use it for uve aggregation DB selection (redis_agg_db offset + 6)
redis_agg_db offset starts from 1
now offset is calcualted based on number of analytics node is installed, once Bug 1784493 is merged,
then we need to use ANALYTICS_ALARM
Related-Bug: #1793847

copy job-manager dependencies to device-manager container
next review will remove it from config-api
Partial-Bug: #1794549

nodemgr: install chrony and ntp to be able to check time sync everywhere
Closes-Bug: #1800704

Add STAT collection destination to the daemon config files
The STAT collector socket path specified in the instances.yaml
have to be populated in the daemon config files.
Closes-Bug: 1794107

Updated mellanox plugin repositories
Partial-bug: #1765858
Signed-off-by: Vladimir Sokolovsky <email address hidden>

Add specific IP option for Introspect in contrail-vcenter-manager config file
Partial-Bug: #1802626

Updated mellanox plugin redhat version to 7.6
Partial-bug: #1765858
Signed-off-by: Matan Azrad <email address hidden>

implement SSL support rabbitmq container
partial-jira-bug: JCB-218684

add container's ID to contrail-status output
closes-jira-bug: JCB-218706

Fixed contrail-status container opts for IPA case
In case of IPA ca file is mounted separately and placed
in /etc/ipa/ca.crt. Node init should skip cert generation,
so is_ssl_enabled is false for it, so, ssl folder is to always mounted.
In case of no ssl it is just empty.
Partial-Bug: #1805443

remove copy-paste from external containers
external containers didn't have access to common definitions and functions.
solution is to copy common.sh and functions into external containers and use
code from it to reduce copy-paste.
closes-jira-bug: JCB-218707

remove call to pre-start-init from externals for now
it waits for certs if SSL_ENABLE is true. but not all containers
have /etc/contrail/ssl mount and some has own flag for SSL.
closes-jira-bug: JCB-218707

implement node-to-node encryption for cassandra
partial-jira-bug: JCB-218753

remove obsolete code and move some code from general-base to specific containers
closes-jira-bug: JCB-218707

use common function in rabbitmq's entrypoint
now rabbitmq has access to common functions and should us them
to avoid copy-paste.
closes-jira-bug: JCB-218707

Added cacert parametet for contrail-status
- in case of IPA cacert is different from
default value
Partial-Bug: #1805443

install pip via easy_install
installing python-pip in RHEL via yum is very ugly.
And anyway after installing pip code upgrades it from pypi.
partial-jira-bug: JCB-218785

create dnsmasq container and remove tftp/dhcp
cleanup obsolete dependencies from config-api (as they moved...

Reviewed: https://review.opencontrail.org/49385
Committed: http://github.com/Juniper/contrail-ansible-deployer/commit/6d56f894b1547baddfc84059e0f6ab3cd34dec8e
Submitter: Vinay Vithal Mahuli (<email address hidden>)
Branch: R6.0-WIP

commit 6d56f894b1547baddfc84059e0f6ab3cd34dec8e
Author: Michael Henkel <email address hidden>
Date: Mon Dec 3 02:49:13 2018 +0100

Apply commits from master onto R6.0-WIP

ansible-deployer use correct InternalIP for kubelet
Closes-Bug: 1806299

Use agent's entrypoint on Windows
Start using agent's entrypoint to achieve similar behaviour
to Linux's microservices.
Partial-Bug: #1806677

Starting docker service before pull
Closes-Bug: #1807716

Closes-Bug: #1808241 - Protect against reprovision failures
VROUTER_NODES_LIST is not needed - causes reprovision un-necessarily
Also, Openstack might be in half-configured state if provision fails first time due to intermittent timeout
Here re-provision shouldn't fail due to missing endpoints from first run.

Support stats collector destination in ansible
Stat messages are written to socket file which will
be read by stats collector. The socket file to be
written to is specified in instances.yaml. The folder
holfing the socket file will be mounted by all the
containers. The user specified STATS_COLLECTOR_DESTINATION_PATH
in the instances.yaml.
Partial-Bug: 1794107

add ssl mounts for rabbitmq and cassandra
closes-jira-bug: JCB-218684

change configuration description for additional rabbitmq options
partial-jira-bug: JCB-218684

Handle domainsuffix for Windows nodes
1. Expose provider_config to install container roles
2. Use proper FQDN in POST to config node
Closes-Bug: #1810173

Add domainsuffix to Windows examples
Closes-Bug: #1810173

Change default openstack version from ocata to queens
Closes Jira Bug: JCB-218736

Use kolla playbook to generate self-signed certs
If generate_self_signed_certs is set to yes under kolla_config->kolla_globals
section of the instance.yml file, then generate self-signed certs to be used for
the openstack haproxy service. Note that this will be used only when
kolla_enable_tls_external is set to true.
If using already generated certs, then give the path to the certificates using
the kolla_external_fqdn_cert variable
Partial Jira Bug: JCB-218738

Making kafka/alarm-gen/snmp-collector/topology/zookeeper/cassandra/query-engine installation optional.
Three new roles are introduced.
analytics_alarm - If added, alarm-gen, kafka will be installed
analytics_snmp - If added, snmp-collector and topology will be installed
If we need to enable analytics_alarm role, then analytics_database_kafka role also needs to
be added in any of the deploying node in cluster.
All components will use config_database zookeeper except kafka.
Partial-Bug: #1784493

add dnsmasq to config-api pod
next commits will move device manager and dnsmasq to separate pod
partial-jira-bug: JCB-218801

add capabilities required for dnsmasq
partial-jira-bug: JCB-218801

Fix extracting hostname from Windows
Use `hostname` bash command instead of ansible_hostname
Closes Jira Bug: TFB-1370

Initial commit for Openstack Rocky support
Partial-Jira-Bug: JCB-218848

fix issues preventing to use...

Review in progress for https://review.opencontrail.org/49583
Submitter: Biswajit Mandal (<email address hidden>)

Reviewed: https://review.opencontrail.org/49583
Committed: http://github.com/Juniper/contrail-ansible-deployer/commit/461eb5e615adab6c2aee5ed556f0c8bec5514904
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 461eb5e615adab6c2aee5ed556f0c8bec5514904
Author: Biswajit Mandal <email address hidden>
Date: Wed Feb 20 23:31:29 2019 +0000

Updating the README.md file as we removed zookeeper from analytics.

Change-Id: I27e51a9822b6b76376efe16c0de9a887002080a7
Closes-Bug: #1784493

Review in progress for https://review.opencontrail.org/49639
Submitter: Biswajit Mandal (<email address hidden>)

Reviewed: https://review.opencontrail.org/49639
Committed: http://github.com/Juniper/contrail-controller/commit/2feae44dcdc46cb97eef11f1765bad626ce8e6b9
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 2feae44dcdc46cb97eef11f1765bad626ce8e6b9
Author: Biswajit Mandal <email address hidden>
Date: Thu Feb 21 20:26:06 2019 +0000

ContrailConfig missing for alarm/snmp node and alarm not raised for components tied in analytics_alarm
and analytics_snmp role

Fixes two issues:
1. Added two roles analytics_alarm and analytics_snmp in _UVEMAP while creating VncDbClient object
2. Added the new roles inside uve_key of alarm_list.

Change-Id: Ia14c8d3b131185e751aeb8189df9bf2b3712e9cf
closes-jira-bug: JCB-219332
Closes-Bug: #1784493

Reviewed: https://review.opencontrail.org/49535
Committed: http://github.com/Juniper/contrail-controller/commit/17c0be3002f723bd589af526f0d9606df0ce7498
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 17c0be3002f723bd589af526f0d9606df0ce7498
Author: Biswajit Mandal <email address hidden>
Date: Tue Feb 19 23:30:23 2019 +0000

While importing cassandra & zookeeper DB through db_json_exim.py, the metadata info is skipped in zookeeper.
So adding /analytics-discovery- also in ignore list.

Change-Id: I84d4cd581b4d696c180e18901da4d78f716f9c96
Closes-Bug: #1784493