Deploying Mirantis OpenStack 6.1 results in network configuration issues with br-aux

Bug #1489683 reported by Bruce Basil Mathews
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
Medium
Igor Zinovik

Bug Description

From a configuration standpoint:
1. Fuel Master is a VMware Virtual Machine
2. The three Controller nodes are three VMware Virtual Machines sitting in separate clusters with compatible IP address schemes.
3. The Compute/Ceph Storage nodes (8 of them total I think) are Bare Metal Servers (Dell 820's). BIOS has been set up properly and the H710 controller has 8 disks each. Two for OS (RAID1), Two for Journal (RAID1), and four for Ceph-OSD.
4. Each Dell has one 1 Gbit NIC for Admin/PXE and a "trunk" of multiple 1 Gbit NICS for the other four networks.

Here is the Network, I setup:

**** Neutron with VLAN segmentation

** Public
IP Range 10.34.84.21 to 10.34.84.100
CIDR 10.34.84.0/22
Use VLAN tagging Yes, 1001
Gateway 10.34.84.1
Floating IP Range 10.34.84.101 to 10.34.84.199

** Storage
CIDR 192.168.1.0/24
Use VLAN tagging Yes, 102

** Management
CIDR 192.168.0.0/24
Use VLAN tagging Yes, 101

** Neutron L2 Configuration
VLAN ID Range 1002 to 1003
Base MAC Address fa:16:3e:00:00:00

** Neutron L3 Configuration
Internal Network CIDR 192.168.111.0/24
Internal Network Gateway 192.168.111.1
Guest OS DNS Servers 10.34.64.11 & 10.34.64.12

Network Verification was successful. However, when a deployment was attempted, Fuel Astute created the br-aux bridge on the root Network card (eth0), which removed br-mgmt, br-ext, etc from the other NICS and the deployment failed. This deployment with the same parameters works as expected when only Bare Metal servers are employed for the Controller nodes.

Revision history for this message
Nastya Urlapova (aurlapova) wrote :

Bruce, please add fuel version and diagnostic snapshot?

Changed in fuel:
assignee: nobody → Fuel Partner Core (fuel-partner-core)
status: New → Incomplete
Revision history for this message
Mark Miller IT Pro (markmilleritpro) wrote :
Revision history for this message
Mark Miller IT Pro (markmilleritpro) wrote :

There has been a zillion reinstalls after that last snapshot. I’ll upload another snapshot, to that same ticket. I’m on the 10th or so iteration of “OS201p"
Now all the nodes are running Ubuntu.

**** Neutron with VLAN segmentation

** Public
IP Range 10.34.88.21 to 10.34.89.220
CIDR 10.34.88.0/22
Use VLAN tagging Yes, 1002
Gateway 10.34.88.1
Floating IP Range 10.34.90.21 to 10.34.91.220

** Storage
CIDR 192.168.1.0/24
Use VLAN tagging Yes, 102

** Management
CIDR 192.168.0.0/24
Use VLAN tagging Yes, 101

** Neutron L2 Configuration
VLAN ID Range 1003 to 1004
Base MAC Address fa:16:3e:00:00:00

** Neutron L3 Configuration
Internal Network CIDR 192.168.111.0/24
Internal Network Gateway 192.168.111.1
Guest OS DNS Servers 10.34.64.11 & 10.34.64.12

Revision history for this message
Bruce Basil Mathews (bmathews-l) wrote :

Please look through new Fuel snapshot once uploaded by Mark Miller.

Revision history for this message
Mark Miller IT Pro (markmilleritpro) wrote :

I split the upload to 3 parts
fuel, controller90, controller92, and the computenodes 84,85,86,87,88,89

The problem is to get a 2nd external vlan 205 10.34.72.0/22 to access DBs on Layer2
+--------------------------------------+-----------+-------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+-----------+-------------------------------------------------------+
| 9a66fc1b-0e12-4f70-9c6f-87b50e13b698 | net04_ext | 7061a3ea-ebd5-414d-8137-c72f6d23ec97 10.34.88.0/22 |
| 00bd4dce-7945-459f-bd83-94b42fb2a7d4 | net04 | 178e3239-8595-4062-9986-72b5df2d5c8c 192.168.111.0/24 |
+--------------------------------------+-----------+-------------------------------------------------------+

Revision history for this message
Mark Miller IT Pro (markmilleritpro) wrote :
Revision history for this message
Mark Miller IT Pro (markmilleritpro) wrote :
Changed in fuel:
assignee: Fuel Partner Core (fuel-partner-core) → Igor Zinovik (izinovik)
Revision history for this message
Igor Zinovik (izinovik) wrote :

Mark, could you please clarify some moments in your setup.

dmidecode.txt files for nodes 90 and 92 show that both of nodes are hardware Dell servers.
In your setup only Fuel Master node operates inside VMware virtual machine, so you
do not use vCenter as hypevisor, only KVM.

During deployment there are no errors in puppet logs.

I draw a diagram of your network setup http://paste.openstack.org/show/475904/

I do not understand about your question access to DB on Layer2.

In your setup each tenant traffic will be tagged (according to settings it will be VLANS 1003 or 1004).
When VM emits traffic compute nodes which hosts this VM will mark this with appropriate
VLAN tag and send it into br-prv, which then will be passed through br-aux and bond0.

When controller receives traffic from compute node it will accept it will pass it through
bond0 - br-aux - br-prv. Neutron L3 agent on controller have to route this traffic out
via br-ex (in you case it is VLAN 1002). For you setup VLAN 1002 is considered as provider
network (this is network 'net04_ext' from tenant perspective).

If you are trying to reach external network with VLAN 205 from VLAN 1002 I suppose that
there is not L3 connectivity in your network infrastructure.

Please clarify what you are trying to achieve.

Revision history for this message
Igor Zinovik (izinovik) wrote :

Attaching network scheme, so it will be easy to reference to it
network_scheme:
   transformations:
   - action: add-br
     name: br-fw-admin
   - action: add-br
     name: br-mgmt
   - action: add-br
     name: br-storage
   - action: add-br
     name: br-ex
   - action: add-br
     name: br-floating
     provider: ovs
   - action: add-patch
     bridges:
     - br-floating
     - br-ex
     provider: ovs
     mtu: 65000
   - action: add-br
     name: br-prv
     provider: ovs
   - action: add-br
     name: br-aux
   - action: add-patch
     bridges:
     - br-prv
     - br-aux
     provider: ovs
     mtu: 65000
   - action: add-port
     bridge: br-fw-admin
     name: eth0
   - bridge: br-aux
     name: bond0
     interfaces:
     - eth1
     - eth2
     - eth4
     - eth5
     - eth6
     bond_properties:
       lacp_rate: fast
       mode: 802.3ad
       xmit_hash_policy: layer2+3
     interface_properties:
       vendor_specific:
         disable_offloading: true
     action: add-bond
   - action: add-port
     bridge: br-storage
     name: bond0.102
   - action: add-port
     bridge: br-mgmt
     name: bond0.101
   - action: add-port
     bridge: br-ex
     name: bond0.1002

Changed in fuel:
importance: Undecided → Medium
milestone: none → 6.1-updates
summary: - Deploying Mirantis OpenStack 6.1 with Controllers on VMware ESXi VMs
- results in network configuration issues with br-aux
+ Deploying Mirantis OpenStack 6.1 results in network configuration issues
+ with br-aux
Revision history for this message
Alexander Kislitsky (akislitsky) wrote :

No updates for 'Incomplete' bug. Moving to 'Invalid'. If you provide more info, please reopen the bug.

Changed in fuel:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.