[RFE] Ironic Neutron ML2 Integration

Specification has been approved and is available at http://specs.openstack.org/openstack/ironic-specs/specs/approved/ironic-ml2-integration.html.

Copying this from the old BP (https://blueprints.launchpad.net/ironic/+spec/ironic-ml2-integration):

Gerrit topic: https://review.openstack.org/#q,topic:bp/ironic-ml2-integration,n,z

Addressed by: https://review.openstack.org/188528
    Initial rough draft of the Ironic ML2 Integration

Merged into ironic-specs:
   https://github.com/openstack/ironic-specs/blob/master/specs/liberty/ironic-ml2-integration.rst

Patches:

Ironic
https://review.openstack.org/#/c/206232/

CLI
https://review.openstack.org/#/c/206144

Nova
https://review.openstack.org/#/c/206163/

Addressed by: https://review.openstack.org/206232
    Addition of portgroups to support LAG interfaces

Addressed by: https://review.openstack.org/206238
    Addition of portgroups to support LAG interfaces

Addressed by: https://review.openstack.org/206243
    Addition of portgroups to support LAG interfaces

Addressed by: https://review.openstack.org/206244
    Addition of portgroups to support LAG interfaces

Addressed by: https://review.openstack.org/206245
    Addition of portgroups to support LAG interfaces

Addressed by: https://review.openstack.org/212064
    Add portgroups to support LAG interfaces - DB

Gerrit topic: https://review.openstack.org/#q,topic:bp/network-provider,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bp/network_provider,n,z

Addressed by: https://review.openstack.org/256363
    Allow to build user image with DIB

Addressed by: https://review.openstack.org/256364
    refactor ironic enroll-node code

Addressed by: https://review.openstack.org/256365
    Add Link-Local-Connection info to ironic port

Addressed by: https://review.openstack.org/256366
    Update Ironic VM network connection

Addressed by: https://review.openstack.org/256367
    Add configure_provision_network function

Fix proposed to branch: master
Review: https://review.openstack.org/263508

Fix proposed to branch: master
Review: https://review.openstack.org/273119

Reviewed: https://review.openstack.org/263508
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=968e8329f4adaee30538a7339f6fb9ca2e239ec3
Submitter: Jenkins
Branch: master

commit 968e8329f4adaee30538a7339f6fb9ca2e239ec3
Author: Devananda van der Veen <email address hidden>
Date: Mon Jan 4 17:12:01 2016 -0800

    devstack 'cleanup-node' script should delete OVS bridges

    When unstack'ing, cleanup-node.sh needs to also remove the ovs bridges
    which were added when creating the node.

    Failing to do this will cause subsequent runs of stack.sh to fail
    because the bridge can't be re-created.

    Change-Id: I5cc36e675d272f6dee3bc667133d1e7e082b0a55
    Partial-bug: #1526403

Reviewed: https://review.openstack.org/206232
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=1740ab79708fa6578db7bf4c607ab046b7fc215d
Submitter: Jenkins
Branch: master

commit 1740ab79708fa6578db7bf4c607ab046b7fc215d
Author: Om Kumar <email address hidden>
Date: Tue Oct 13 19:46:15 2015 +0530

    Add portgroups to support LAG interfaces - DB

    Ironic should be able to provide the requisite connectivity
    information to the Neutron ML2 plugin to allow drivers to
    provision the top-of-rack switch for the bare metal server.
    The addition of portgroups in Ironic allows the concept of
    link aggregation to be handled in Ironic in order to provide
    support for cases where multiple interfaces on the bare metal
    server connect to switch ports of a single LAG.

    This commit includes changes to:
    - the DB models (extension of port model and addition of portgroup
      model)
    - the DB tests

    Partial-bug: #1526403
    DocImpact
    Co-Authored-By: Laura Moore (<email address hidden>)
    Co-Authored-By: Jenny Moorehead (<email address hidden>)
    Co-Authored-By: Will Stevenson (<email address hidden>)

    Change-Id: Ic028c316b0670c2d18de89111e83069f3441f476

Reviewed: https://review.openstack.org/206238
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=134ad4cbf6f1930de9c4dd7fd653fedb7decaf18
Submitter: Jenkins
Branch: master

commit 134ad4cbf6f1930de9c4dd7fd653fedb7decaf18
Author: Laura Moore <email address hidden>
Date: Mon Jul 27 17:01:19 2015 -0400

    Add portgroups to support LAG interfaces - objs

    Ironic should be able to provide the requisite connectivity
    information to the Neutron ML2 plugin to allow drivers to
    provision the top-of-rack switch for the bare metal server.
    The addition of portgroups in Ironic allows the concept of
    link aggregation to be handled in Ironic in order to provide
    support for cases where multiple interfaces on the bare metal
    server connect to switch ports of a single LAG.

    This commit includes changes to:
    - the objects (extension of port and addition of portgroup)
    - the object tests

    Partial-bug: #1526403
    DocImpact
    Co-Authored-By: Jenny Moorehead (<email address hidden>)
    Co-Authored-By: Will Stevenson (<email address hidden>)

    Change-Id: Id13eeafbdd9a4d9e679121f1b60be0073430499b

Reviewed: https://review.openstack.org/206243
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=a603e3c12c62e29e12e45283d8602f9aa5acfc0f
Submitter: Jenkins
Branch: master

commit a603e3c12c62e29e12e45283d8602f9aa5acfc0f
Author: Laura Moore <email address hidden>
Date: Mon Jul 27 17:14:40 2015 -0400

    Add portgroups to support LAG interfaces - RPC

    Ironic should be able to provide the requisite connectivity
    information to the Neutron ML2 plugin to allow drivers to
    provision the top-of-rack switch for the bare metal server.
    The addition of portgroups in Ironic allows the concept of
    link aggregation to be handled in Ironic in order to provide
    support for cases where multiple interfaces on the bare metal
    server connect to switch ports of a single LAG.

    This commit includes changes to:
    - the RPC API (to include portgroup methods)
    - the RPC tests

    Partial-bug: #1526403
    DocImpact
    Co-Authored-By: Jenny Moorehead (<email address hidden>)
    Co-Authored-By: Will Stevenson (<email address hidden>)

    Change-Id: Iac6355e615e57e6ed002facd6795c9ff06188998

Reviewed: https://review.openstack.org/206245
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=e5c5ddbdc8b015221b2656270a2f3f21414a055f
Submitter: Jenkins
Branch: master

commit e5c5ddbdc8b015221b2656270a2f3f21414a055f
Author: Laura Moore <email address hidden>
Date: Mon Jul 27 17:33:22 2015 -0400

    Add portgroups to support LAG interfaces - net

    Ironic should be able to provide the requisite connectivity
    information to the Neutron ML2 plugin to allow drivers to
    provision the top-of-rack switch for the bare metal server.
    The addition of portgroups in Ironic allows the concept of
    link aggregation to be handled in Ironic in order to provide
    support for cases where multiple interfaces on the bare metal
    server connect to switch ports of a single LAG.

    This commit includes changes to:
    - the port vif mapping to also use portgroups

    Partial-bug: #1526403
    DocImpact
    Co-Authored-By: Jenny Moorehead (<email address hidden>)
    Co-Authored-By: Will Stevenson (<email address hidden>)
    Co-Authored-By: Vasyl Saienko (<email address hidden>)

    Change-Id: I7f1cba65275078da750aa49ae83ba7345d6fd5e0

Change abandoned by Jim Rollenhagen (<email address hidden>) on branch: master
Review: https://review.openstack.org/273119
Reason: Abandoning as this has been rolled into the larger API patch.

Fix proposed to branch: master
Review: https://review.openstack.org/293520

Change abandoned by Vladyslav Drok (<email address hidden>) on branch: master
Review: https://review.openstack.org/139687
Reason: New patch is here - https://review.openstack.org/#/c/285852/

Fix proposed to branch: master
Review: https://review.openstack.org/317392

Fix proposed to branch: master
Review: https://review.openstack.org/317393

Fix proposed to branch: master
Review: https://review.openstack.org/322796

Fix proposed to branch: master
Review: https://review.openstack.org/332177

Reviewed: https://review.openstack.org/256365
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=b9cf7d42b1dd861bfd812de3928f58fe6aa46ce8
Submitter: Jenkins
Branch: master

commit b9cf7d42b1dd861bfd812de3928f58fe6aa46ce8
Author: vsaienko <email address hidden>
Date: Fri Dec 11 11:41:00 2015 +0200

    Add Link-Local-Connection info to ironic port

    This patch introduce new variable called IRONIC_USE_LINK_LOCAL.
    Link-Local-Connection information is used by Neutron to bind_port
    on the switch.

    Change-Id: Icfb9257bded5b234a9d05f2b2de6d6580f2f42aa
    Partial-bug: #1526403

Reviewed: https://review.openstack.org/317391
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=b186297f324231db43f77999359714c67aa45973
Submitter: Jenkins
Branch: master

commit b186297f324231db43f77999359714c67aa45973
Author: Vasyl Saienko <email address hidden>
Date: Tue May 17 13:59:24 2016 +0300

    Add network_interface node field to DB and object

    This patch adds a new field 'network_interface' to node object.
    Its default value is None meaning 'use the default network interface'
    to comply with driver composition reform. The default setting logic
    will be added to the driver factory in the next patch.

    Partial-bug: #1526403
    Co-Authored-By: Vladyslav Drok <email address hidden>
    Change-Id: Id9b9f4199c2640ca30b2d87c3a517c65f9cb5527

Reviewed: https://review.openstack.org/338417
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=0a5bb693efe6a5b417c5ba9a912292b498438f84
Submitter: Jenkins
Branch: master

commit 0a5bb693efe6a5b417c5ba9a912292b498438f84
Author: Vladyslav Drok <email address hidden>
Date: Wed Jul 6 20:32:15 2016 +0300

    Add internal_info field to ports and portgroups

    In case of ports, it is also added to the API, as a readonly field.
    It will be used for any port-specific internal information ironic
    needs to store inside the port object. In this change we start using
    it to store UUIDs of the cleaning ports that ironic creates, instead
    of fiddling with port.extra['vif_port_id'], as extra is intended for
    operator use only.

    Partial-bug: #1526403

    Change-Id: Ib62c3e32305619d0c55f8ec7e45b067f0f0b32d4

Reviewed: https://review.openstack.org/285852
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=cde11611d93946a1c79e406dc429aa5e742da729
Submitter: Jenkins
Branch: master

commit cde11611d93946a1c79e406dc429aa5e742da729
Author: Vasyl Saienko <email address hidden>
Date: Tue May 17 13:59:39 2016 +0300

    Add network interface to base driver class

    This change also introduces two network interfaces:

    * flat: Copies current neutron DHCP provider logic to work with
      cleaning ports;
    * noop: noop interface.

    The default value of the network_interface is None, meaning that the
    node will be using the default network interface. The default network
    interface is determined the following way:

    * if [DEFAULT]default_network_interface configuration option is set
      (the default for it is None), the specified interface becomes the
      default for all nodes;

    * if it is not set, 'flat' interface will be used if the deployment
      currently uses 'neutron' DHCP provider, otherwise 'noop' interface
      will be used.

    create_cleaning_ports and delete_cleaning_ports methods of the DHCP
    providers are still being called in case of out-of-tree DHCP
    providers, but this possibility will be removed completely in the
    next release. If the DHCP provider logic is rewritten into a custom
    network interface, please remove those methods from the provider, so
    that network interface is called instead.

    Partial-bug: #1526403
    Co-Authored-By: Om Kumar <email address hidden>
    Co-Authored-By: Vasyl Saienko <email address hidden>
    Co-Authored-By: Sivaramakrishna Garimella <email address hidden>
    Co-Authored-By: Vladyslav Drok <email address hidden>
    Co-Authored-By: Zhenguo Niu <email address hidden>
    Change-Id: I0c26582b6b6e9d32650ff3e2b9a3269c3c2d5454

Reviewed: https://review.openstack.org/293520
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=58fbfabedaa18418415957b9f754c204dcceb3f1
Submitter: Jenkins
Branch: master

commit 58fbfabedaa18418415957b9f754c204dcceb3f1
Author: vsaienko <email address hidden>
Date: Wed Mar 16 15:16:28 2016 +0200

    Allow to use network interfaces in devstack

    This patch allow to set enabled_network_interfaces configuration option
    via IRONIC_ENABLED_NETWORK_INTERFACES and registered ironic node with
    IRONIC_NETWORK_INTERFACE.

    Change-Id: Ie01ca4be3a085bc283879b5528c5c174a35a843b
    Depends-On: I0c26582b6b6e9d32650ff3e2b9a3269c3c2d5454
    Partial-bug: #1526403

Reviewed: https://review.openstack.org/317393
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=ab97fa0f1f9ed6acb78c1ceadec8b333eac73254
Submitter: Jenkins
Branch: master

commit ab97fa0f1f9ed6acb78c1ceadec8b333eac73254
Author: Vasyl Saienko <email address hidden>
Date: Tue May 17 14:00:00 2016 +0300

    Add 'neutron' network interface

    This patch adds a 'neutron' network interface. This interface
    supports separate networks for provisioning and for cleaning of nodes.

    Partial-bug: #1526403
    Co-Authored-By: Vladyslav Drok <email address hidden>
    Change-Id: Ia3442ab3536a1a8d8839b24dbfc640b818450350

Reviewed: https://review.openstack.org/213262
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=5041703401ded33fc0b54409d60667900a319403
Submitter: Jenkins
Branch: master

commit 5041703401ded33fc0b54409d60667900a319403
Author: Om Kumar <email address hidden>
Date: Fri Aug 14 22:56:24 2015 +0530

    Update the deploy drivers with network flipping logic

    This patch consumes the plumbing work for network interfaces and thus
    provides support for tenant network isolation for baremetal.

    Partial-bug: #1526403
    Co-Authored-By: Vasyl Saienko <email address hidden>
    Co-Authored-By: Vladyslav Drok <email address hidden>
    Co-Authored-By: Sivaramakrishna Garimella <email address hidden>

    Change-Id: I2f0a508973ff2f674c16f91d5dc440f9e8e667bf

Reviewed: https://review.openstack.org/206244
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=0a5549680d491894faad5133db38e1bc2b4c93d6
Submitter: Jenkins
Branch: master

commit 0a5549680d491894faad5133db38e1bc2b4c93d6
Author: Laura Moore <email address hidden>
Date: Mon Jul 27 17:29:57 2015 -0400

    Add multitenancy-related fields to port API object

    This commit adds new fields to port object:
    - port.pxe_enabled: indicates whether pxe is enabled or disabled
          for this port
    - port.local_link_connection: contains the port binding profile.

    Partial-bug: #1526403
    Co-Authored-By: Jenny Moorehead <email address hidden>
    Co-Authored-By: Will Stevenson <email address hidden>
    Co-Authored-By: Vasyl Saienko <email address hidden>
    Co-Authored-By: Vladyslav Drok <email address hidden>
    Co-Authored-By: Zhenguo Niu <email address hidden>

    Change-Id: Ie655fd59b06de7b84fba3b438d5e4c2ecd8075c3

Reviewed: https://review.openstack.org/256366
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=8fe320753c10ede6445781cc884cadc2bc235ea5
Submitter: Jenkins
Branch: master

commit 8fe320753c10ede6445781cc884cadc2bc235ea5
Author: vsaienko <email address hidden>
Date: Fri Dec 11 12:05:59 2015 +0200

    Update Ironic VM network connection

    When VM is dirrectly connected to OVS (current setup) there is no port
    in OVS when VM is in powered off state. Since Ironic plug port to
    network when VM is in powered-off state. Port should be present in OVS
    even when VM is powered-off.

    This patch adds additional network bridge called br-$VM_NAME for each
    VM. Adds additional interface ovs-$VM_NAME which is added to both OVS
    and br-$VM_NAME. And configures VM NIC in br-$VM_NAME instead of direct
    connection to OVS switch.

    Now connection looks like:

                   Linux Bridge OVS
     -------- ----------- ---------
     | vm-1 | <--> | br-vm-1 | <---> | br-bm |
     -------- ----------- ---------

    Change-Id: I317dd9429684a14d16ecf2887d45c2b014e36a84
    Partial-bug: #1526403

Reviewed: https://review.openstack.org/317392
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=c62e1bee29e795ece05b6be8c613910d75a7543e
Submitter: Jenkins
Branch: master

commit c62e1bee29e795ece05b6be8c613910d75a7543e
Author: Vasyl Saienko <email address hidden>
Date: Tue May 17 13:59:50 2016 +0300

    Expose node's network_interface field in API

    This patch exposes the node's network_interface field in the REST API.
    It also adds restrictions on the node states in which network
    interface change is possible and whether the requested network
    interface is enabled.

    As a temporary solution until the driver composition work is completed,
    we have taken an approach that requires all API and Conductor nodes to
    have the same setting for enabled_network_interfaces. There are inline
    notes in the code indicating where we will address this in the future.

    Partial-bug: #1526403
    Co-Authored-By: Om Kumar <email address hidden>
    Co-Authored-By: Vasyl Saienko <email address hidden>
    Co-Authored-By: Sivaramakrishna Garimella <email address hidden>
    Co-Authored-By: Vladyslav Drok <email address hidden>
    Co-Authored-By: Zhenguo Niu <email address hidden>
    Change-Id: I67495196c3334f51ed034f4ca6e32a3e01a58f15

Reviewed: https://review.openstack.org/256367
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=75fc071b54a07fbdbaabaebd359299170bd7fda2
Submitter: Jenkins
Branch: master

commit 75fc071b54a07fbdbaabaebd359299170bd7fda2
Author: vsaienko <email address hidden>
Date: Fri Dec 11 13:14:00 2015 +0200

    Add configure_provision_network function

    This change allows to configure ironic provision network by setting
    IRONIC_PROVISION_NETWORK_NAME variable. In this case additional
    interface $OVS_PHYSICAL_BRIDGE.$IRONIC_PROVISION_SEGMENTATION_ID
    will be configured with IRONIC_PROVISION_SUBNET_GATEWAY ip address.

    Additional configuration parameters are:

    IRONIC_PROVISION_PROVIDER_NETWORK_TYPE
    IRONIC_PROVISION_SEGMENTATION_ID
    IRONIC_PROVISION_ALLOCATION_POOL
    IRONIC_PROVISION_SUBNET_GATEWAY
    IRONIC_PROVISION_SUBNET_PREFIX

    Change-Id: I619f2fb92aafe7348b0a47eaaaad1790df5ae5c7
    Partial-bug: #1526403

Reviewed: https://review.openstack.org/258596
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=ad5a06c53422bf9872e884974461cb5a307bd20c
Submitter: Jenkins
Branch: master

commit ad5a06c53422bf9872e884974461cb5a307bd20c
Author: vsaienko <email address hidden>
Date: Wed Dec 16 17:53:58 2015 +0200

    Add multitenancy devstack configuration examples.

    The guide shows how to setup Devstack with Ironic multitenancy
    and ``networking-generic-switch`` Neutron ML2 driver.

    Depends-On: If2fb996783b9ac26a5bae2aadd6387207750def9
    Change-Id: I24ef394536e5109cf1499ec87f6a495063a87f5d
    Closes-bug: #1526403

Fix proposed to branch: master
Review: https://review.openstack.org/347549

Reviewed: https://review.openstack.org/325299
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=395069cbef7f92e0ee191320f626a32756e6835b
Submitter: Jenkins
Branch: master

commit 395069cbef7f92e0ee191320f626a32756e6835b
Author: Vladyslav Drok <email address hidden>
Date: Fri Jun 3 16:32:55 2016 +0300

    Add api-ref for new port fields

    API microversions 1.18, 1.19 introduced new fields in Port resource,
    this change adds them to api-ref documentation.

    Partial-bug: #1526403

    Change-Id: Ib7059fd1ec8ffc54fa34647cc558fb8c8e80f8d7

This issue was fixed in the openstack/ironic 6.1.0 release.

Reviewed: https://review.openstack.org/228496
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=6d323b6f55e5f4b9be0228b14d13fdc8d5fc2d13
Submitter: Jenkins
Branch: master

commit 6d323b6f55e5f4b9be0228b14d13fdc8d5fc2d13
Author: Carmelo Ragusa <email address hidden>
Date: Mon Sep 28 16:20:58 2015 +0100

    Operator documentation for multitenancy

    Partial-bug: #1526403
    Co-Authored-By: Vladyslav Drok <email address hidden>
    Change-Id: Id59616d8b3ac4a4ebae01012db6e1f1923a8ed78

Change abandoned by Vladyslav Drok (<email address hidden>) on branch: master
Review: https://review.openstack.org/322796

Reviewed: https://review.openstack.org/360647
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=c8d518898e27951123ad7816337d2a0f862ea5b4
Submitter: Jenkins
Branch: master

commit c8d518898e27951123ad7816337d2a0f862ea5b4
Author: Vasyl Saienko <email address hidden>
Date: Thu Aug 25 11:25:32 2016 -0400

    Rely on portgroup standalone_ports_supported

    Some hardware doesn't support portgroup fallback to single interface
    fashion. This imposes additional restrictions on ports, for example
    such ports will not support booting by PXE.
    This patch adds restrictions setting pxe_enabled=True on ports that
    are members of portgroups with standalone_ports_supported=False. And
    vice versa portgroup.standalone_ports_supported can't be set to False
    until portgroup contains ports with pxe_enabled=True.

    Setting 'vif_port_id' means that we are using this port in standalone
    mode. This patch also ensures that 'vif_port_id' can't be updated on
    ports which are members of portgroups with
    standalone_ports_supported=False. And vice versa updating portgroup
    standalone_ports_supported=False is not allowed if portgroup contains
    ports with extra/vif_port_id set.

    Partial-bug: #1526403

    Change-Id: I9b4682918725bed2da0b7c89666e2c37d8826290