openstack-ansible

Issues with neutron metadata service in AIO

Series trunk
Bug #1483603

Bug #1483603 reported by Matt Thompson on 2015-08-11

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
openstack-ansible	Fix Released	High	Matt Thompson
Kilo	Fix Released	High	Jesse Pretorius	openstack-ansible 11.1.2
Trunk	Fix Released	High	Matt Thompson

Bug Description

While working on https://review.openstack.org/#/c/195403/ (disabling force_config_drive for live migrations), I noticed that instances on an AIO cannot reach the neutron metadata service when force_config_drive is set to False. Testing this on physical hardware with the same neutron network layout (that laid down by tempest) did not yield the same results.

Ideally, we will want to default force_config_drive to False when we bump to liberty, however we will need to get the neutron metadata service working in an AIO environment first.

See original description

Matt Thompson (mattt416) on 2015-08-11

description:

updated

Matt Thompson (mattt416) on 2015-08-11

Changed in openstack-ansible:
importance:	Undecided → High
status:	New → Confirmed

Revision history for this message

Matt Thompson (mattt416) wrote on 2015-08-11:

Apsu investigated this for us, and found that in the AIO we'll need to drop the following in any neutron_agents containers:

iptables -t mangle -A POSTROUTING -j CHECKSUM --checksum-fill

Some chat log for posterity:

16:30:08 Apsu | Instantly worked.
16:30:33 Apsu | The symptom is that the vxlan iface gets the 169.254.169.254:80 traffic but the bridge it's in does not
16:30:39 Apsu | Which happens due to the invalid checksum.
16:30:53 Apsu | Which happens because there's not a real network interface managing checksums
16:31:18 Apsu | On a multi-node, that instance traffic crosses a real interface on the way across the vxlan tunnel
16:31:23 Apsu | So it picks up a proper checksum

Revision history for this message

Jesse Pretorius (jesse-pretorius) wrote on 2015-08-11:

To be more precise, it should be: iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill

Revision history for this message

Matt Thompson (mattt416) wrote on 2015-08-13:

Master review here: https://review.openstack.org/#/c/211918/

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-15: Fix merged to os-ansible-deployment (master)

Reviewed: https://review.openstack.org/211918
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=d879994e13e788123e2db1368b985d0e5fe9cb1b
Submitter: Jenkins
Branch: master

commit d879994e13e788123e2db1368b985d0e5fe9cb1b
Author: Matt Thompson <email address hidden>
Date: Wed Aug 12 10:12:57 2015 +0100

Add iptables rule to neutron agents containers

    When running in an AIO environment, we need to drop an iptables rule to
    ensure that communication between instances and the neutron metadata
    service works.

Change-Id: Icc081fe83712ce883baa88f99db60c52dcc4c1ae
Closes-Bug: #1483603

Changed in openstack-ansible:
status:	Triaged → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-15: Fix proposed to os-ansible-deployment (kilo)

Fix proposed to branch: kilo
Review: https://review.openstack.org/213442

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-17: Fix merged to os-ansible-deployment (kilo)

Reviewed: https://review.openstack.org/213442
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=e7d7494f1aaf20bff3dba9203b906b55e0bf0102
Submitter: Jenkins
Branch: kilo

commit e7d7494f1aaf20bff3dba9203b906b55e0bf0102
Author: Matt Thompson <email address hidden>
Date: Wed Aug 12 10:12:57 2015 +0100

Add iptables rule to neutron agents containers

    When running in an AIO environment, we need to drop an iptables rule to
    ensure that communication between instances and the neutron metadata
    service works.

    Change-Id: Icc081fe83712ce883baa88f99db60c52dcc4c1ae
    Closes-Bug: #1483603
    (cherry picked from commit d879994e13e788123e2db1368b985d0e5fe9cb1b)