packet droped by tap service's qbr bridge

Bug #1870083 reported by Wei Hui
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tap-as-a-service
In Progress
Undecided
Unassigned

Bug Description

steps to reproduce:

create 3 ports use the same vlan network as port_src, port_dst, port_target.
create 3 vms use those 3 ports as vm_src, vm_dst, vm_target.
vm_src on compute1, vm_dst and vm_target on compute2.
vm_src ping vm_dst OK
vm_src ping vm_target OK
vm_dst ping vm_target OK
create tap service use port port_target
create tap flow use port port_src
then vm_src ping vm_dst

expected result:
vm_src and vm_dst icmp packet was mirrored to vm_target

actual result:
vm_target tcpdump not show icmp packet

vm_target connect to qbr_vm_target, then use veth pair connect qbr_vm_target and br-int
tap_vm_target<---qbr_vm_target--->qvb_vm_target------------qvo_vm_target<----br-int
qbr_vm_target is linux bridge, learning mac then forward, it learn's vm_src and vm_dst macs on port qvb_vm_target, when mirrored packet reach qbr_vm_target from port qvb_vm_target, it drops packet.

Revision history for this message
Lajos Katona (lajos-katona) wrote :

I create the same setup and for me tcpdump sowed icmp.
I used vxlan network:
$ openstack network create tap_net --provider-network-type vxlan --provider-segment 1000
....
I have the 3 ports.
...
I have ubuntu16.04 for target vm, and cirros for dst and src.

ubuntu@vm-target:~$ sudo tcpdump -nni -vvv -i ens3 icmp

sudo: unable to resolve host vm-target: Connection timed out
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes
14:53:58.601256 IP 192.168.103.9 > 192.168.103.22: ICMP echo request, id 44289, seq 35, length 64
14:53:58.602412 IP 192.168.103.22 > 192.168.103.9: ICMP echo reply, id 44289, seq 35, length 64
14:53:59.173048 IP 192.168.103.22 > 192.168.103.9: ICMP echo request, id 45569, seq 39, length 64
14:53:59.173560 IP 192.168.103.9 > 192.168.103.22: ICMP echo reply, id 45569, seq 39, length 64
14:53:59.602423 IP 192.168.103.9 > 192.168.103.22: ICMP echo request, id 44289, seq 36, length 64
14:53:59.603810 IP 192.168.103.22 > 192.168.103.9: ICMP echo reply, id 44289, seq 36, length 64

Revision history for this message
Lajos Katona (lajos-katona) wrote :

I checked the dump a little more and for me the host local VLAN is in the mirrored traffic. Would be good if someone else could validate it.
Anyway I check with your patch

Revision history for this message
Lajos Katona (lajos-katona) wrote :
Revision history for this message
Wei Hui (huiweics) wrote :

Thanks for your work.
Can you check this way.

On vm_target, open a terminal, ping vm_src, open another terminal, ping vm_dst, keep pinging.
meanwhile vm_src ping vm_dst.

linux bridge learning mac, when a packet arrive, check mac table then forward. if dst mac not in mac table, it flood the packet all ports except in port. Learned mac not stay in mac table forever, a aging timer count down, when time out, learned mac delete.

In your environment, I think linux bridge does not learn mac or learned mac time out. linux bridge use flood to forward mirrored packet.

Revision history for this message
Bence Romsics (bence-romsics) wrote :

There may be another option. Since you have qbr and qvo devices in your report I assume you use the hybrid firewall driver. But since the openvswitch firewall driver became available there's no advantage of the hybrid driver over the ovs fw driver anymore. Therefore with all bugs involving the hybrid driver it is recommended to check if the same bug can be reproduced with the ovs fw driver. If the bug is specific to the hybrid driver then switch to the ovs fw driver. And you won't lose anything. Actually you may gain some better performance.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tap-as-a-service (master)

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tap-as-a-service/+/711885
Reason: If needed, feel free to restore the patch

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.