iptable rules collision deployed with k8s iptables kube-proxy enabled

Bug #1908957 reported by norman shen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
norman shen

Bug Description

Maybe it's a k8s kube-proxy related bug, but maybe it is easier to solve on neutron's side...

In k8s either NodePort or ExternalIP will generate iptable rules which will effect vm traffic when
hybrid iptable plugin enabled.

The problem is:

Chain PREROUTING (policy ACCEPT 650 packets, 65873 bytes)
 pkts bytes target prot opt in out source destination
 560K 37M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-is-in
  56M 4944M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
  40M 3785M KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */

And packets will be DNAT to something which we do not want and such traffic will be dropped in the end.

By adding the following rule it seems problem is mitigated,

iptables -t nat -I PREROUTING 2 -m physdev --physdev-is-in -j ACCEPT

Changed in neutron:
importance: Undecided → High
norman shen (jshen28)
Changed in neutron:
assignee: nobody → norman shen (jshen28)
Revision history for this message
Lajos Katona (lajos-katona) wrote :
Changed in neutron:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/768322
Committed: https://opendev.org/openstack/neutron/commit/08032e9cc65fe79a53a217b6f061af745ee374b8
Submitter: "Zuul (22348)"
Branch: master

commit 08032e9cc65fe79a53a217b6f061af745ee374b8
Author: shenjiatong <email address hidden>
Date: Tue Dec 22 09:01:50 2020 +0800

    Allow neutron managed ports to bypass PREROUTING chain

    When deployed with k8s, k8s service types like NodePort
    or ExternalIP will affect vm traffic on nat table's
    PREROUTING chain. This PS try to mitigate the effect
    by allowing vm traffic to bypass those rules.

    Change-Id: Iae12d9c2f37bc0fca9c3d5e85e46c642263e4a77
    Closes-Bug: #1908957

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 24.0.0.0b1

This issue was fixed in the openstack/neutron 24.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.