Bug #1781121 “Vcenter only-provisioning : Need to have consisten...” : Bugs : Juniper Openstack

Revision history for this message

amudhar (amudha) wrote on 2018-07-11:

#1

Download full text (11.1 KiB)

From: Amudha R <email address hidden>
Date: Wednesday, July 11, 2018 at 1:05 AM
To: Sandip Dey <email address hidden>, Sachchidanand Vaidya <email address hidden>, Ram Yadav <email address hidden>
Cc: Sudheendra Rao <email address hidden>
Subject: Re: R5.0.1 vcenter provisioning

Please read inline.

Thanks,
Amudha

From: Sandip Dey <email address hidden>
Date: Tuesday, July 10, 2018 at 11:36 AM
To: Amudha R <email address hidden>, Sachchidanand Vaidya <email address hidden>, Ram Yadav <email address hidden>
Cc: Sudheendra Rao <email address hidden>
Subject: Re: R5.0.1 vcenter provisioning

My setup is up with sriov/pcie/ctrl-data single interface and eam.

You can use if you want to check things

10.204.216.61 - controller
All the files are here /root/contrail-ansible-deployer

Amudha, the pcie interface came up with name eth0 this time.
[Amudha] - Did you mean eth20?

We need to fix the interface name something consistent for all kind of configuration of the ctrl-data interface.
[Amudha] - ctrl-data interface is a vmxnet3 interface and comes up as ens*. In case of PCI and SR-IOV interfaces,
they are ixgbe interfaces and will come up as eth* interfaces.

I have a script to give ip to the ctrl-data interface.If the the name is not same for all the compute-vms,script will fail.
[Amudha] Can your script check for ens and eth* as well? And may be use the driver type for additional check if needed?..

Also will write a script to create bond .I would assume the interfaces names to be eth20 and eth21 for bond.
[Amudha] Should work with the current vmdk.

The vmwarephysicalinterface should also ideally be same for any kind of configuration for ctrl-data.That was the case for fab.It used to be always ‘eth1’
[Amudha] I am not sure about this.. Would think it to be eth2 in case of ctrl-data interface defined.

We have to get rid of the situation where vmwarephysicalinterface=ens192 for some case and vmwarephysicalinterface=ens224 for some other.
[Amudha] Each scenario creates different interfaces, and this is hidden from user. Would not prefer changing this in 5.0.1 beta.
We can have a discussion once all are available, and see if it really needs changes.

Regards
Sandip

From: Sandip Dey <email address hidden>
Date: Tuesday, July 10, 2018 at 7:51 PM
To: Amudha R <email address hidden>, Sachchidanand Vaidya <email address hidden>, Ram Yadav <email address hidden>
Cc: Sudheendra Rao <email address hidden>
Subject: Re: R5.0.1 vcenter provisioning

Hi Amudha

Please see inline….

Ram,

What is the behavior if we create a pg in the vm dvs from vcenter ui for which vn is not there in api server?Does the vcenter-plugin delete it since, its not there in the api server?

I wanted to delete the contrail-vm/vm dvs from vcenter and rerun the vcenter provisioning when rest of the contrail components are up and running.I see the dvs pg on vm dvs getting deleted as soon as it was created.
I am not sure if the vcenter plugin deleting it,because there is not explicit log for deletion.I see only this.

Any port group created on that vm dvs is getting deleted.

[2018-07-10 12:50:01,070] [INFO ] [Thread-3:VCenterDB@794] [Found <dpg c4k4u14_dvpg, dvs c4k4_dvs, datacenter c4_da...

From: Amudha R <amudha@juniper.net>
Date: Wednesday, July 11, 2018 at 1:05 AM
To: Sandip Dey <sandipd@juniper.net>, Sachchidanand Vaidya <vaidyasd@juniper.net>, Ram Yadav <ryadav@juniper.net>
Cc: Sudheendra Rao <sudheendra@juniper.net>
Subject: Re: R5.0.1 vcenter provisioning

Please read inline.

Thanks,
Amudha

From: Sandip Dey <sandipd@juniper.net>
Date: Tuesday, July 10, 2018 at 11:36 AM
To: Amudha R <amudha@juniper.net>, Sachchidanand Vaidya <vaidyasd@juniper.net>, Ram Yadav <ryadav@juniper.net>
Cc: Sudheendra Rao <sudheendra@juniper.net>
Subject: Re: R5.0.1 vcenter provisioning

My setup is up with sriov/pcie/ctrl-data single interface and eam.

You can use if you want to check things

10.204.216.61 - controller
All the files are here /root/contrail-ansible-deployer

Amudha, the pcie interface came up with name eth0 this time. 
[Amudha] - Did you mean eth20?

We need to fix the interface name something consistent for all kind of configuration of the ctrl-data interface.
[Amudha] - ctrl-data interface is a vmxnet3 interface and comes up as ens*. In case of PCI and SR-IOV interfaces,
they are ixgbe interfaces and will come up as eth* interfaces.

I have a script to give ip to the ctrl-data interface.If the the name  is not same for all the compute-vms,script will fail.
[Amudha] Can your script check for ens and eth* as well? And may be use the driver type for additional check if needed?..

Also will write a script to create bond .I would assume the interfaces names to be eth20 and eth21 for bond.
[Amudha] Should work with the current vmdk.

The vmwarephysicalinterface should also ideally be same for any kind of configuration for ctrl-data.That was the case for fab.It used to be always ‘eth1’
[Amudha] I am not sure about this.. Would think it to be eth2 in case of ctrl-data interface defined.

We have to get rid of the situation where vmwarephysicalinterface=ens192 for some case and vmwarephysicalinterface=ens224 for some other.
[Amudha] Each scenario creates different interfaces, and this is hidden from user. Would not prefer changing this in 5.0.1 beta. 
We can have a discussion once all are available, and see if it really needs changes.

Regards
Sandip

From: Sandip Dey <sandipd@juniper.net>
Date: Tuesday, July 10, 2018 at 7:51 PM
To: Amudha R <amudha@juniper.net>, Sachchidanand Vaidya <vaidyasd@juniper.net>, Ram Yadav <ryadav@juniper.net>
Cc: Sudheendra Rao <sudheendra@juniper.net>
Subject: Re: R5.0.1 vcenter provisioning

Hi Amudha

Please see inline….

Ram,

What is the behavior if we create a pg in the vm dvs from vcenter ui for which vn is not there in api server?Does the vcenter-plugin delete it since, its not there in the api server?

I wanted to delete the contrail-vm/vm dvs from vcenter and rerun the vcenter provisioning when rest of the contrail components are up and running.I see the dvs pg on vm dvs getting deleted as soon as it was created.
I am not sure if the vcenter plugin deleting it,because there is not explicit log for deletion.I see only this.

Any port group created on that vm dvs is getting deleted.

[2018-07-10 12:50:01,070] [INFO ] [Thread-3:VCenterDB@794] [Found <dpg c4k4u14_dvpg, dvs c4k4_dvs, datacenter c4_datacenter11, vCenter https://10.204.217.246/sdk>.]
[2018-07-10 12:50:06,075] [INFO ] [Thread-3:VCenterNotify@664] [+++++++++++++ Done syncing +++++++++++++++++++++]
[2018-07-10 12:50:06,075] [INFO ] [Thread-3:VCenterNotify@667] [+++++++++++++ Waiting for events +++++++++++++++++++++]
[2018-07-10 12:50:06,075] [INFO ] [Thread-3:VncDB@155] [ Checking if api-server is alive and kicking..]
[2018-07-10 12:50:06,075] [INFO ] [Thread-3:ApiConnectorImpl@217] [>> Request: GET, /projects]
[2018-07-10 12:50:06,078] [INFO ] [Thread-3:ApiConnectorImpl@239] [<< Response Status: HTTP/1.1 200 OK]
[2018-07-10 12:50:06,080] [INFO ] [Thread-3:VncDB@171] [ Api-server alive. Got the pulse..]
[2018-07-10 12:50:06,080] [INFO ] [Thread-3:VCenterNotify@642] [+++++++++++++ Start syncing  +++++++++++++++++++++]
[2018-07-10 12:50:06,080] [INFO ] [Thread-3:VncDB@813] [Start reading virtual networks from the API server ...]
[2018-07-10 12:50:06,080] [INFO ] [Thread-3:ApiConnectorImpl@217] [>> Request: GET, /virtual-networks]
[2018-07-10 12:50:06,084] [INFO ] [Thread-3:ApiConnectorImpl@239] [<< Response Status: HTTP/1.1 200 OK]
[2018-07-10 12:50:06,086] [INFO ] [Thread-3:ApiConnectorImpl@217] [>> Request: GET, /virtual-network/2abc2b5c-8da9-4227-acc8-dd8f949891d2]
[2018-07-10 12:50:06,089] [INFO ] [Thread-3:ApiConnectorImpl@239] [<< Response Status: HTTP/1.1 200 OK]
[2018-07-10 12:50:06,091] [INFO ] [Thread-3:ApiConnectorImpl@217] [>> Request: GET, /virtual-network/007f8dd7-8bc5-42ef-8036-eac1374701c9]
[2018-07-10 12:50:06,094] [INFO ] [Thread-3:ApiConnectorImpl@239] [<< Response Status: HTTP/1.1 200 OK]
[2018-07-10 12:50:06,096] [INFO ] [Thread-3:ApiConnectorImpl@217] [>> Request: GET, /virtual-network/a9ba499e-dd29-4dd9-ac53-9a283b668526]
[2018-07-10 12:50:06,101] [INFO ] [Thread-3:ApiConnectorImpl@239] [<< Response Status: HTTP/1.1 200 OK]
[2018-07-10 12:50:06,103] [INFO ] [Thread-3:ApiConnectorImpl@217] [>> Request: GET, /virtual-network/33a11739-d562-4329-9abe-946d55c1d7b0]
[2018-07-10 12:50:06,107] [INFO ] [Thread-3:ApiConnectorImpl@239] [<< Response Status: HTTP/1.1 200 OK]
[2018-07-10 12:50:06,109] [INFO ] [Thread-3:ApiConnectorImpl@217] [>> Request: GET, /virtual-network/24da76ed-9534-47bb-b396-f4c8d2134af0]
[2018-07-10 12:50:06,112] [INFO ] [Thread-3:ApiConnectorImpl@239] [<< Response Status: HTTP/1.1 200 OK]
[2018-07-10 12:50:06,114] [INFO ] [Thread-3:VncDB@846] [Done reading from the API server, found 2 Virtual Networks]
[2018-07-10 12:50:06,114] [INFO ] [Thread-3:VCenterDB@885] [Start reading virtual networks from vcenter ...]
[2018-07-10 12:50:06,134] [ERROR] [Thread-3:VCenterDB@688] [dvSwitch: c4k4_dvs Datacenter: c4_datacenter11 Private VLAN NOTconfigured]
[2018-07-10 12:50:06,134] [ERROR] [Thread-3:VirtualNetworkInfo@277] [VN <ctest-vCenter-55191957, f90106d9-fc85-3c37-a483-bc78ae8fa5a6>Cannot populate vlan, private vlan not configured on dvSwitch: c4k4_dvs Datacenter: c4_datacenter11]
[2018-07-10 12:50:06,140] [INFO ] [Thread-3:VirtualNetworkInfo@401] [Set ipPoolId to null for VN <ctest-vCenter-55191957, f90106d9-fc85-3c37-a483-bc78ae8fa5a6>]
[2018-07-10 12:50:06,140] [INFO ] [Thread-3:VCenterDB@926] [Read from vcenter VN <ctest-vCenter-55191957, f90106d9-fc85-3c37-a483-bc78ae8fa5a6>, ipPoolId null]
[2018-07-10 12:50:06,144] [ERROR] [Thread-3:VCenterDB@688] [dvSwitch: c4k4_dvs Datacenter: c4_datacenter11 Private VLAN NOTconfigured]
[2018-07-10 12:50:06,144] [ERROR] [Thread-3:VirtualNetworkInfo@277] [VN <c4k4u14_dvpg, 75acac93-6416-38b5-9c24-1b75c0adbea3>Cannot populate vlan, private vlan not configured on dvSwitch: c4k4_dvs Datacenter: c4_datacenter11]

Regards
Sandip
From: Amudha R <amudha@juniper.net>
Date: Tuesday, July 10, 2018 at 6:40 PM
To: Sandip Dey <sandipd@juniper.net>, Sachchidanand Vaidya <vaidyasd@juniper.net>, Ram Yadav <ryadav@juniper.net>
Cc: Sudheendra Rao <sudheendra@juniper.net>
Subject: Re: R5.0.1 vcenter provisioning

Sandip,

For 2. Even with single-interface and multi-interface – the vmware_physical_interface would be different – ens192 for single-intf and ens224 for multi-intf.

We have become aware of this value and worry about it because of the issues we had with 5.0 GA build – with the issues fixed and provisioning populating the
vmware_physical_interface to correct value, this is hidden from user..

+++sandipd: I am more concern explaining things to our guys.When things are fine, no issues.But if something goes wrong, finding the correct depending on the setup will be difficult.
                       In fab , we had consistent interface for vmwarephysical interface

The PCI and SR-IOV interfaces are renamed by a script /firstboot in the ContrailVM during bootup..

+++sandipd:For pcie, the interface rename not happening looks like.

-Amudha

From: Sandip Dey <sandipd@juniper.net>
Date: Tuesday, July 10, 2018 at 7:56 AM
To: Amudha R <amudha@juniper.net>, Sachchidanand Vaidya <vaidyasd@juniper.net>, Ram Yadav <ryadav@juniper.net>
Cc: Sudheendra Rao <sudheendra@juniper.net>
Subject: Re: R5.0.1 vcenter provisioning

Hi Amudha

My concern is

1.interfaces are coming up with different names for different contrail_vms
2.Again vmwarephysicalinterface was different (for some ens192 or ens224) in different agents.

Do you think theses are problems?At least #2 is a problem to me.It is really confusing.

Regards
Sandip

From: Amudha R <amudha@juniper.net>
Date: Tuesday, July 10, 2018 at 5:12 PM
To: Sandip Dey <sandipd@juniper.net>, Sachchidanand Vaidya <vaidyasd@juniper.net>, Ram Yadav <ryadav@juniper.net>
Cc: Sudheendra Rao <sudheendra@juniper.net>
Subject: Re: R5.0.1 vcenter provisioning

PCI and SR-IOV interfaces, both should be renamed as eth20, eth21 and so on.
Like we discussed, this was to avoid confusion while finding the vmware_physical_interface.

vrouter:
        PHYSICAL_INTERFACE:

PHYSICAL_INTERFACE is optional and CONTROL_NODES can be specified to deduce the physical interface.
And this is populated in instances.yaml after the ContrailVM is up and the interfaces are setup,
and the users would be aware of the interfaces configured on the ContrailVM. 
vmware_physical_interface is populated in contrail-vrouter-agent.conf by provisioning and is from the user.

bms2:
    provider: bms
    esxi_host: 10.84.24.4
    ip: 10.84.24.26
    roles:
      vrouter:
  bms3:
    provider: bms
    esxi_host: 10.84.24.7
    ip: 10.84.24.235
    roles:
      vrouter:

global_configuration:
  CONTAINER_REGISTRY: opencontrailnightly
  CONTRAIL_VERSION: 5.0-94

contrail_configuration:
  CLOUD_ORCHESTRATOR: vcenter
  CONTROLLER_NODES: 10.84.13.14
  CONTROL_NODES: 192.168.0.241

There were issues when I tried renaming the PCI and SRIOV interfaces to ens* interfaces.
I will again check the script and update here.

Please update once you provision contrail and test functionality.

Thanks,
Amudha

From: Sandip Dey <sandipd@juniper.net>
Date: Tuesday, July 10, 2018 at 2:53 AM
To: Sachchidanand Vaidya <vaidyasd@juniper.net>, Ram Yadav <ryadav@juniper.net>, Amudha R <amudha@juniper.net>
Cc: Sudheendra Rao <sudheendra@juniper.net>
Subject: R5.0.1 vcenter provisioning

Hi Amudha

I am able to bring up the contrail-vm.Yet to provision the contrail.I see these issues

ISSUES
====== 
Problem 1:

1.SRIOV interface names : eth20,eth21
2.single ctrl-data ens192
3.pcie ens160f0,ens160f1

Should we have a consistent naming?

Problem 2
=========
1.pcie enabled vm - ens192 connected to vm-dvs
2.sriov enabled vm - ens192 connected to vm-dvs
3.single ctrl-data interface - ens224 connected to vm-dvs and ens192 connected to ctrl-data-dvs

Again inconsistency. Should not we create consistent interface connecting to vm-dvs.The above behavior will definitely confuse users while making the instance.yml

bms4:
    provider: bms
    esxi_host: 10.204.216.224
    ip: 10.204.216.181
    roles:
      vrouter:
        PHYSICAL_INTERFACE: ens192     >>>> this entry we do for the ctrl-data interface.Now it will be different for single ctrl/pcie /sriov
  bms5:
    provider: bms
    esxi_host: 10.204.216.225
    ip: 10.204.216.182
    roles:
      vrouter:
        PHYSICAL_INTERFACE: ens192

Let me know your thoughts.Will raise bugs accordingly.

Regards
Sandip

Revision history for this message

amudhar (amudha) wrote on 2018-07-11:

#2

3.pcie pass through ctrl-data interface names ens160f0,ens160f1 (sometimes eth0 in when re-ran the provisioning)

This shouldn't be the case, and does not happen on my ContrailVMs deployed on two different
ESXi hosts.

Revision history for this message

Sandip Dey (sandipd) wrote on 2018-07-12:

#3

today I brought up again with build 139.
Pcie setup got interface 'eth0'

[sandipd@nodem4 ~]$ ssh root@10.204.216.183
root@10.204.216.183's password:
Last login: Wed Jul 4 22:30:28 2018 from 172.29.108.219
[root@nodek6-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.183 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::9288:aba4:d2e0:e6e prefixlen 64 scopeid 0x20<link>
        inet6 fe80::56da:6476:c3b1:4d9 prefixlen 64 scopeid 0x20<link>
        inet6 fe80::8afe:b1fc:ae8c:3231 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:05 txqueuelen 1000 (Ethernet)
        RX packets 3988 bytes 265441 (259.2 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 128 bytes 14155 (13.8 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:50:56:bb:bb:04 txqueuelen 1000 (Ethernet)
        RX packets 2 bytes 120 (120.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 48 bytes 7968 (7.7 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:25:90:c7:7f:f4 txqueuelen 1000 (Ethernet)
        RX packets 49 bytes 13656 (13.3 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 46 bytes 7284 (7.1 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
        device memory 0xfd400000-fd4fffff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
        inet 127.0.0.1 netmask 255.0.0.0
        inet6 ::1 prefixlen 128 scopeid 0x10<host>
        loop txqueuelen 1000 (Local Loopback)
        RX packets 0 bytes 0 (0.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 0 bytes 0 (0.0 B)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Revision history for this message

Sandip Dey (sandipd) wrote on 2018-07-12:

#4

Download full text (3.4 KiB)

My setup is up with sriov/pcie/ctrl-data single interface and eam.

You can use if you want to check things

10.204.216.61 - controller
All the files are here /root/contrail-ansible-deployer

Amudha, the pcie interface came up with name eth0 this time.
[Amudha] - Did you mean eth20?
+++sandipd: It was eth0.When I brought up 2 pcie interfaces, they were ens160f0,ens160f1

ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.183 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::250:56ff:feaa:aa05 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:05 txqueuelen 1000 (Ethernet)
        RX packets 1690602 bytes 579191961 (552.3 MiB)
        RX errors 0 dropped 2 overruns 0 frame 0
        TX packets 171108 bytes 232670877 (221.8 MiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens192: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
        inet6 fe80::250:56ff:febb:bb04 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:bb:bb:04 txqueuelen 1000 (Ethernet)
        RX packets 207 bytes 66846 (65.2 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 8 bytes 648 (648.0 B)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet6 fe80::225:90ff:fec7:7ff4 prefixlen 64 scopeid 0x20<link>
        ether 00:25:90:c7:7f:f4 txqueuelen 1000 (Ethernet)
        RX packets 127288 bytes 8523996 (8.1 MiB)
        RX errors 0 dropped 17 overruns 0 frame 0
        TX packets 140608 bytes 8934212 (8.5 MiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
        device memory 0xfd400000-fd4fffff

We need to fix the interface name something consistent for all kind of configuration of the ctrl-data interface.
[Amudha] - ctrl-data interface is a vmxnet3 interface and comes up as ens*. In case of PCI and SR-IOV interfaces,
they are ixgbe interfaces and will come up as eth* interfaces.

I have a script to give ip to the ctrl-data interface.If the the name is not same for all the compute-vms,script will fail.
[Amudha] Can your script check for ens and eth* as well? And may be use the driver type for additional check if needed?..

Also will write a script to create bond .I would assume the interfaces names to be eth20 and eth21 for bond.
[Amudha] Should work with the current vmdk.

The vmwarephysicalinterface should also ideally be same for any kind of configuration for ctrl-data.That was the case for fab.It used to be always ‘eth1’
[Amudha] I am not sure about this.. Would think it to be eth2 in case of ctrl-data interface defined.

+++sandipd:It was always ‘eth1’.Please check the fab code and document attached

We have to get rid of the situation where vmwarephysicalinterface=ens192 for some case and vmwarephysicalinterface=ens224 for some other.
[Amudha] Each scenario creates different interfaces, and this is hidden from user. Would not prefer changing this in 5.0.1 beta.
We can have a discussion once all are available, and see if it really needs changes.

+++sandipd:'Each scenario creates different interfaces’.This is th...

My setup is up with sriov/pcie/ctrl-data single interface and eam.

You can use if you want to check things

10.204.216.61 - controller
All the files are here /root/contrail-ansible-deployer

Amudha, the pcie interface came up with name eth0 this time. 
[Amudha] - Did you mean eth20?
+++sandipd: It was eth0.When I brought up 2 pcie interfaces, they were ens160f0,ens160f1

ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.204.216.183  netmask 255.255.255.0  broadcast 10.204.216.255
        inet6 fe80::250:56ff:feaa:aa05  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:aa:aa:05  txqueuelen 1000  (Ethernet)
        RX packets 1690602  bytes 579191961 (552.3 MiB)
        RX errors 0  dropped 2  overruns 0  frame 0
        TX packets 171108  bytes 232670877 (221.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens192: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        inet6 fe80::250:56ff:febb:bb04  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:bb:bb:04  txqueuelen 1000  (Ethernet)
        RX packets 207  bytes 66846 (65.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::225:90ff:fec7:7ff4  prefixlen 64  scopeid 0x20<link>
        ether 00:25:90:c7:7f:f4  txqueuelen 1000  (Ethernet)
        RX packets 127288  bytes 8523996 (8.1 MiB)
        RX errors 0  dropped 17  overruns 0  frame 0
        TX packets 140608  bytes 8934212 (8.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xfd400000-fd4fffff

We need to fix the interface name something consistent for all kind of configuration of the ctrl-data interface.
[Amudha] - ctrl-data interface is a vmxnet3 interface and comes up as ens*. In case of PCI and SR-IOV interfaces,
they are ixgbe interfaces and will come up as eth* interfaces.

I have a script to give ip to the ctrl-data interface.If the the name  is not same for all the compute-vms,script will fail.
[Amudha] Can your script check for ens and eth* as well? And may be use the driver type for additional check if needed?..

Also will write a script to create bond .I would assume the interfaces names to be eth20 and eth21 for bond.
[Amudha] Should work with the current vmdk.

The vmwarephysicalinterface should also ideally be same for any kind of configuration for ctrl-data.That was the case for fab.It used to be always ‘eth1’
[Amudha] I am not sure about this.. Would think it to be eth2 in case of ctrl-data interface defined.

+++sandipd:It was always ‘eth1’.Please check the fab code and document attached

We have to get rid of the situation where vmwarephysicalinterface=ens192 for some case and vmwarephysicalinterface=ens224 for some other.
[Amudha] Each scenario creates different interfaces, and this is hidden from user. Would not prefer changing this in 5.0.1 beta. 
We can have a discussion once all are available, and see if it really needs changes.

+++sandipd:'Each scenario creates different interfaces’.This is the confusion and difficult to remember.We would definitely will have hard time debugging things going down the line.
                       We wont remember what is the interface in what scenario.Yes, I leave it to Sachin to decide on it.

Regards
Sandip

Sudheendra Rao (sudheendra-k) on 2018-07-16

information type:

Proprietary → Public

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2018-08-10: [Review update] master

#5

Review in progress for https://review.opencontrail.org/45469
Submitter: amudha ramachandran (<email address hidden>)

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2018-08-10: [Review update] R5.0

#9

Review in progress for https://review.opencontrail.org/45470
Submitter: amudha ramachandran (<email address hidden>)

Revision history for this message

amudhar (amudha) wrote on 2018-08-10:

#11

- Fix involves - not executing /firstboot at ContrailVM creation,
this will result in the following names for the ContrailVM

mgmt ctrl-data vmware_physical_intf

single-intf ens160 -- ens192

multi-intf ens160 ens192 ens224
(ctrl-data)

multi-intf ens160 ens192 ens224
(pci/sriov)

This change is done and saved in the ContrailVM vmdk at:
/cs-shared/contrail-vcenter/vmdk/centos-7.5/LATEST

- When finding vmware_physical_interface, exclude PCI/SRIOV
interfaces (driver_type should be vmxnet3)
https://review.opencontrail.org/#/c/45469/

Revision history for this message

Sandip Dey (sandipd) wrote on 2018-08-10:

#12

Download full text (6.5 KiB)

Hi Amudha

I took the latest ovf from the specified location.

Ran ansible-playbook playbooks/vcenter.yml . There is still something missing.Please fix it.

[root@nodek4-compute-vm ~]#
[root@nodek4-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet6 fe80::9288:aba4:d2e0:e6e prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:a4:92:2a txqueuelen 1000 (Ethernet)
        RX packets 38 bytes 9120 (8.9 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 54 bytes 8996 (8.7 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.181 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::a0af:9125:f772:13ad prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:03 txqueuelen 1000 (Ethernet)
        RX packets 476 bytes 44433 (43.3 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 73 bytes 9173 (8.9 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet6 fe80::9ee2:b14c:219:186b prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:a4:20:56 txqueuelen 1000 (Ethernet)
        RX packets 0 bytes 0 (0.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 51 bytes 8994 (8.7 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
        inet 127.0.0.1 netmask 255.0.0.0
        inet6 ::1 prefixlen 128 scopeid 0x10<host>
        loop txqueuelen 1000 (Local Loopback)
        RX packets 32 bytes 2592 (2.5 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 32 bytes 2592 (2.5 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[root@nodek4-compute-vm ~]#

[root@nodek6-compute-vm ~]#
[root@nodek6-compute-vm ~]#
[root@nodek6-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:25:90:c7:7f:f4 txqueuelen 1000 (Ethernet)
        RX packets 44 bytes 11540 (11.2 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 59 bytes 9290 (9.0 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
        device memory 0xfd400000-fd4fffff

ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.183 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::3a64:a693:b05:b908 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:05 txqueuelen 1000 (Ethernet)
        RX packets 654 bytes 57651 (56.2 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 76 bytes 9519 (9.2 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:50:56:a4:f6:c6 txqueuelen 1000 (Ethernet)
        RX packets 0 bytes 0 (0.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 50 bytes 8652 (8.4 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 col...

Hi Amudha

I took the latest ovf from the specified location.

Ran ansible-playbook playbooks/vcenter.yml . There is still something missing.Please fix it.

[root@nodek4-compute-vm ~]# 
[root@nodek4-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::9288:aba4:d2e0:e6e  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:a4:92:2a  txqueuelen 1000  (Ethernet)
        RX packets 38  bytes 9120 (8.9 KiB)
        RX errors 0  dropped 1  overruns 0  frame 0
        TX packets 54  bytes 8996 (8.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.204.216.181  netmask 255.255.255.0  broadcast 10.204.216.255
        inet6 fe80::a0af:9125:f772:13ad  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:aa:aa:03  txqueuelen 1000  (Ethernet)
        RX packets 476  bytes 44433 (43.3 KiB)
        RX errors 0  dropped 1  overruns 0  frame 0
        TX packets 73  bytes 9173 (8.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::9ee2:b14c:219:186b  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:a4:20:56  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 51  bytes 8994 (8.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 32  bytes 2592 (2.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 32  bytes 2592 (2.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@nodek4-compute-vm ~]#

[root@nodek6-compute-vm ~]# 
[root@nodek6-compute-vm ~]# 
[root@nodek6-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:25:90:c7:7f:f4  txqueuelen 1000  (Ethernet)
        RX packets 44  bytes 11540 (11.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 59  bytes 9290 (9.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xfd400000-fd4fffff

ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.204.216.183  netmask 255.255.255.0  broadcast 10.204.216.255
        inet6 fe80::3a64:a693:b05:b908  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:aa:aa:05  txqueuelen 1000  (Ethernet)
        RX packets 654  bytes 57651 (56.2 KiB)
        RX errors 0  dropped 1  overruns 0  frame 0
        TX packets 76  bytes 9519 (9.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:50:56:a4:f6:c6  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 50  bytes 8652 (8.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 32  bytes 2592 (2.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 32  bytes 2592 (2.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@nodek6-compute-vm ~]#

From: Amudha R <amudha@juniper.net>
Date: Friday, August 10, 2018 at 10:23 AM
To: Sandip Dey <sandipd@juniper.net>, Sudheendra Rao <sudheendra@juniper.net>
Cc: Sachchidanand Vaidya <vaidyasd@juniper.net>, Yuvaraja Mariappan <ymariappan@juniper.net>
Subject: Re: 5.0.1 - #1782515 - ContrailVM interface name change on reboot

Yes, it is the same vmdk.

-Amudha

From: Sandip Dey <sandipd@juniper.net>
Date: Thursday, August 9, 2018 at 11:38 PM
To: Amudha R <amudha@juniper.net>, Sudheendra Rao <sudheendra@juniper.net>
Cc: Sachchidanand Vaidya <vaidyasd@juniper.net>, Yuvaraja Mariappan <ymariappan@juniper.net>
Subject: Re: 5.0.1 - #1782515 - ContrailVM interface name change on reboot

Can the same image be used for ram also?I am planning to test everything using eam now

From: Sandip Dey <sandipd@juniper.net>
Date: Friday, August 10, 2018 at 10:05 AM
To: Amudha R <amudha@juniper.net>, Sudheendra Rao <sudheendra@juniper.net>
Cc: Sachchidanand Vaidya <vaidyasd@juniper.net>, Yuvaraja Mariappan <ymariappan@juniper.net>
Subject: Re: 5.0.1 - #1782515 - ContrailVM interface name change on reboot

This was exactly what was needed.

What if there are multiple-ctrl interface ?In case of bond we need it.What will be the naming?

mgmt                    ctrl-data                vmware_phy_int
  
single-intf           ens160                 —                            ens192    >>>>>> can we make it ‘ens224’ to be consistent in all the cases?

Regards
Sandip

From: Amudha R <amudha@juniper.net>
Date: Friday, August 10, 2018 at 10:02 AM
To: Sandip Dey <sandipd@juniper.net>, Sudheendra Rao <sudheendra@juniper.net>
Cc: Sachchidanand Vaidya <vaidyasd@juniper.net>, Yuvaraja Mariappan <ymariappan@juniper.net>
Subject: Re: 5.0.1 - #1782515 - ContrailVM interface name change on reboot

https://bugs.launchpad.net/juniperopenstack/+bug/1782515
https://bugs.launchpad.net/juniperopenstack/+bug/1781121

For the above two bugs,

- I have removed the /firstboot script which was renaming the PCI/SRIOV interfaces, 
  and saved the vmdk at /cs-shared/contrail-vcenter/vmdk/centos-7.5/LATEST.
  Note: This was added in ubuntu to keep interface names consistent across reboots,
  and does not seem needed in centos7.5. Have tested the latest vmdk and the 
  interface names are consistent across reboots.
  
- Check for driver type to be vmxnet3 while picking the vmware_physical_nterface,
  fixed in https://review.opencontrail.org/#/c/45470/ .

With this, the interfaces will come up as below:

mgmt                    ctrl-data                vmware_phy_int
  
single-intf           ens160                 —                            ens192

multi-intf            ens160                  ens192                   ens224
(ctrl-data)

multi-intf            ens160                  ens192                   ens224
(pci/sriov)

Please test with the latest vmdk and the review above merged, and let me know of any issues.

Thanks,
Amudha

Sandip Dey (sandipd) on 2018-08-10

tags:

added: blocker

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2018-08-11: A change has been merged

#13

Reviewed: https://review.opencontrail.org/45469
Committed: http://github.com/Juniper/contrail-container-builder/commit/123d46112cce27bdf3ff467589a02cda3dcf191f
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 123d46112cce27bdf3ff467589a02cda3dcf191f
Author: Amudha <email address hidden>
Date: Thu Aug 9 20:05:32 2018 -0700

vCenter: Fix ContrailVM interface names

- Fix involves - not executing /firstboot at ContrailVM creation,
this will result in the following names for the ContrailVM

mgmt ctrl-data vmware_physical_intf

single-intf ens160 -- ens192

multi-intf ens160 ens192 ens224
(ctrl-data)

multi-intf ens160 ens192 ens224
(pci/sriov)

This change is done and saved in the ContrailVM vmdk.

- When finding vmware_physical_interface, exclude PCI/SRIOV
interfaces (driver_type should be vmxnet3)

Change-Id: Iffed0cd9125cd98e5b232ce30428f3ece6a19986
Closes-Bug: #1781121

Sudheendra Rao (sudheendra-k) on 2018-08-12

tags:

added: sanity-blocker
removed: blocker

Sudheendra Rao (sudheendra-k) on 2018-08-12

tags:

added: sanityblocker
removed: sanity-blocker

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2018-08-12:

#14

Reviewed: https://review.opencontrail.org/45470
Committed: http://github.com/Juniper/contrail-container-builder/commit/22259f0a38eb77b29351533dd956891025336daa
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 22259f0a38eb77b29351533dd956891025336daa
Author: Amudha <email address hidden>
Date: Thu Aug 9 20:05:32 2018 -0700

vCenter: Fix ContrailVM interface names

- Fix involves - not executing /firstboot at ContrailVM creation,
this will result in the following names for the ContrailVM

mgmt ctrl-data vmware_physical_intf

single-intf ens160 -- ens192

multi-intf ens160 ens192 ens224
(ctrl-data)

multi-intf ens160 ens192 ens224
(pci/sriov)

This change is done and saved in the ContrailVM vmdk.

- When finding vmware_physical_interface, exclude PCI/SRIOV
interfaces (driver_type should be vmxnet3)

Change-Id: Iffed0cd9125cd98e5b232ce30428f3ece6a19986
Closes-Bug: #1781121

Revision history for this message

amudhar (amudha) wrote on 2018-08-13:

#15

mgmt
----
- The mgmt interface will be ens160.

ctrl/data
---------
- The ctrl-data interface will be the next added interface - ens192, and add on as ens224, etc.
- In case of PCI/SRIOV, interfaces will start at ens320, and add on as ens352 and so on.

vmware_physical_interface
-------------------------
- In any setup, the last available vmxnet3 interface will be the vmware_physical_interface.

mgmt ctrl-data vmware_phy_int
single-intf ens160 --- ens192
multi-intf ens160 ens192 ens224
multi-intf bond ens160 ens192,ens224 ens256
pci/sriov ens160 ens320 ens192
pci-sriov bond ens160 ens320,ens352 ens192

	Status	Importance	Assigned to	Milestone
Juniper Openstack	Status tracked in Trunk
R5.0	Fix Committed	Medium	amudhar	Juniper Openstack r5.0.1
Trunk	Fix Committed	Medium	amudhar	Juniper Openstack r5.1.0

Juniper Openstack

Vcenter only-provisioning : Need to have consistent naming for the contrail-vm interface

Bug Description

Other bug subscribers

Remote bug watches