neutron

[OVN] VMs cannot access metadata when connected to a network with only IPv6 subnets

Bug #2069482 reported by Miguel Lavalle
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Miguel Lavalle

Bug Description

VMs cannot access the metadata service when connected to a network with only IPv6 subnets.

Neutron branch: master

Steps to reproduce:

1) Create a network with a single IPv6 subnet:

$ openstack network create ipv6-net-dhcpv6-slaac
$ openstack subnet create --subnet-range fdba:e036:9e22::/64 --ip-version 6 --gateway dba:e036:9e22::1 --ipv6-ra-mode slaac --ipv6-address-mode slaac --network ipv6-net-dhcpv6-slaac ipv6-subnet-dhcpv6-slaac

2) Create a VM using this network:

$ openstack server create --key-name my_key --flavor m1.small --image ubuntu-20.04-minimal-cloudimg-amd64 --network ipv6-net-dhcpv6-slaac --security-group sg1 my-vm-slaac

3) The following message is added to the metadata agent log file:

Jun 14 22:00:32 central neutron-ovn-metadata-agent[89379]: DEBUG neutron.agent.ovn.metadata.agent [-] No valid VIF ports were found for network 191a0539-edbc-4037-b973-dfa77e3208f6, tearing the namespace down if needed {{(pid=89379) _get_provision_params /opt/stack/neutron/neutron/agent/ovn/metadata/agent.py:720}}

which is produced here:

https://github.com/openstack/neutron/blob/79b2d709c80217830fed8ad73dcf6fbd3eea91b4/neutron/agent/ovn/metadata/agent.py#L719-L723

4) When an IPv4 subnet is added to the network and the VM is recreated, the metadata service is accessible to it over IPv6:

$ openstack subnet create --network ipv6-net-dhcpv6-slaac ipv4-subnet-dhcpv6-slaac --subnet-range 10.2.0.0/24
$ openstack server delete my-vm-slaac
$ openstack server create --key-name my_key --flavor m1.small --image ubuntu-20.04-minimal-cloudimg-amd64 --network ipv6-net-dhcpv6-slaac --security-group sg1 my-vm-slaac

From the VM:

ubuntu@my-vm-slaac:~$ curl http://[fe80::a9fe:a9fe%ens3]
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04
latest

ubuntu@my-vm-slaac:~$ curl http://[fe80::a9fe:a9fe%ens3]/openstack
2012-08-10
2013-04-04
2013-10-17
2015-10-15
2016-06-30
2016-10-06
2017-02-22
2018-08-27
2020-10-14
latest

How reproducible: 100%

Tags: ipv6 ovn
Miguel Lavalle (minsel)
Changed in neutron:
importance: Undecided → High
assignee: nobody → Miguel Lavalle (minsel)
Oleg Bondarev (obondarev)
Changed in neutron:
status: New → Triaged
tags: added: ovn
Dr. Jens Harbott (j-harbott)
tags: added: ipv6
OpenStack Infra (hudson-openstack)
Changed in neutron:
status: Triaged → In Progress
Revision history for this message
Alex Bazhaninovskiy (alexbazhan1) wrote :

Hello. We faced the same problem. We found that when creating a new interface for metadata in a function "_get_provision_params" only ipv4 port addresses are collected by "_get_port_ip4_ips". Why is only ipv4 used in this place? We offer a small patch that allows to collect ipv6 too.

Revision history for this message
Miguel Lavalle (minsel) wrote :

This is the fix I'm proposing: https://review.opendev.org/c/openstack/neutron/+/922264

Revision history for this message
Alex Bazhaninovskiy (alexbazhan1) wrote :

Why should the "_get_provision_params" function return a list of only ipv4 addresses, and not all ipv4+ipv6? Could this cause any problems?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron-tempest-plugin (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/925928

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/926503

Miguel Lavalle (minsel)
summary: [OVN] VMs cannot access metadata when connected to a network with only
- IPv6 subnets
+ IPv6 subnets in the Neutron gate
summary: [OVN] VMs cannot access metadata when connected to a network with only
- IPv6 subnets in the Neutron gate
+ IPv6 subnets
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/922264
Committed: https://opendev.org/openstack/neutron/commit/f7000f3d57bc59732522c4943d6ff2e9dfcf7d31
Submitter: "Zuul (22348)"
Branch: master

commit f7000f3d57bc59732522c4943d6ff2e9dfcf7d31
Author: Miguel Lavalle <email address hidden>
Date: Tue Jun 18 19:36:13 2024 -0500

    Fix support of IPv6 only networks in OVN metadata agent

    When an IPv6 only network is used as the sole network for a VM and
    there are no other bound ports on the same network in the same chassis,
    the OVN metadata agent concludes that the associated namespace is not
    needed and deletes it. As a consequence, the VM cannot access the
    metadata service. With this change, the namespace is preserved if there
    is at least one bound port on the chassis with either IPv4 or IPv6
    addresses.

    Closes-Bug: #2069482

    Change-Id: Ie15c3344161ad521bf10b98303c7bb730351e2d8

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron-tempest-plugin (master)

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/925928
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/e0d03fc6b2a3a6b3ff89f07d5aa0501daa0a76eb
Submitter: "Zuul (22348)"
Branch: master

commit e0d03fc6b2a3a6b3ff89f07d5aa0501daa0a76eb
Author: Miguel Lavalle <email address hidden>
Date: Wed Aug 7 19:27:25 2024 -0500

    Test metadata query over IPv6 only network

    A test case is added to test querying the metadata service over an IPv6
    only network

    Depends-On: https://review.opendev.org/c/openstack/neutron/+/922264
    Change-Id: I7db8b7cbd199fc15ecc3b28fe95e041c5957c574
    Related-Bug: #2069482
    Related-Bug: #2076916

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/2024.1)

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/neutron/+/927321

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/2023.2)

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/neutron/+/927322

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/2023.2)

Change abandoned by "Miro Tomaska <email address hidden>" on branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/neutron/+/927322
Reason: This fix no longer needed since the feature backport(this is fixing) to a stable branch is not permitted.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/2024.1)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/927321
Committed: https://opendev.org/openstack/neutron/commit/b98003cf9fe756af331a20a48740138e7cde0366
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit b98003cf9fe756af331a20a48740138e7cde0366
Author: Miguel Lavalle <email address hidden>
Date: Tue Jun 18 19:36:13 2024 -0500

    Fix support of IPv6 only networks in OVN metadata agent

    When an IPv6 only network is used as the sole network for a VM and
    there are no other bound ports on the same network in the same chassis,
    the OVN metadata agent concludes that the associated namespace is not
    needed and deletes it. As a consequence, the VM cannot access the
    metadata service. With this change, the namespace is preserved if there
    is at least one bound port on the chassis with either IPv4 or IPv6
    addresses.

    Closes-Bug: #2069482

    Change-Id: Ie15c3344161ad521bf10b98303c7bb730351e2d8
    (cherry picked from commit f7000f3d57bc59732522c4943d6ff2e9dfcf7d31)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron-tempest-plugin (master)

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/926503
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/4a0b2343d723ea1227e85e0776fc58988a6b9e35
Submitter: "Zuul (22348)"
Branch: master

commit 4a0b2343d723ea1227e85e0776fc58988a6b9e35
Author: Miguel Lavalle <email address hidden>
Date: Sun Aug 18 17:20:51 2024 -0500

    Test metadata query over IPv6 only network with OVS and LB

    This change enables the testing of querying the metadata service over an
    IPv6 only network

    Depends-On: https://review.opendev.org/c/openstack/neutron/+/922264

    Change-Id: I56b1b7e5ca69e2fb01d359ab302e676773966aca
    Related-Bug: #2069482
    Closes-Bug: 2076916

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 25.0.0.0rc1

This issue was fixed in the openstack/neutron 25.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 24.1.0

This issue was fixed in the openstack/neutron 24.1.0 Caracal release.

Revision history for this message
Alan Baghumian (alanbach) wrote :

Would it be possible to back port this to Jammy/Yoga?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.