Inconsistent behavior in NIC RSS queue loading

Bug #1940928 reported by Diko Parvanov
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

Utilizing kvms with ubuntu we face this inconsistent issue where an instance only uses a single rx queue (4 vCPUs):

ethtool -S ens7 | egrep 'rx-.\.|rx_discards: '
rx_discards: 0
rx-0.packets: 2653705823
rx-0.bytes: 561319217396
rx-1.packets: 0
rx-1.bytes: 0
rx-2.packets: 0
rx-2.bytes: 0
rx-3.packets: 0
rx-3.bytes: 0

building another identical instances however sometimes works on all 4 (4 vCPUs):

ethtool -S ens6 | egrep 'rx-.\.|rx_discards: '
rx_discards: 0
rx-0.packets: 234661700
rx-0.bytes: 49678565503
rx-1.packets: 231886603
rx-1.bytes: 49022714157
rx-2.packets: 232424910
rx-2.bytes: 49313057910
rx-3.packets: 235776722
rx-3.bytes: 49863841179

Applying the following workaround to the non-working instance:

ethtool -L ens7 combined 2
ethtool -L ens7 combined 4

makes all RSS queues work properly (4). This looks like a race condition during boot or a kernel bug.

The kernel used during the tests is:
5.4.0-81-generic #91-Ubuntu SMP Thu Jul 15 19:09:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
---
ProblemType: Bug
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Aug 24 13:23 seq
 crw-rw---- 1 root audio 116, 33 Aug 24 13:23 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.18
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
CasperMD5CheckResult: skip
DistroRelease: Ubuntu 20.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Lsusb-t: /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
MachineType: OpenStack Foundation OpenStack Nova
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=vt220
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.4.0-81-generic root=UUID=c77fa46f-24bd-4fdb-8dd9-a1d1a98b3ecf ro console=tty1 console=ttyS0
ProcVersionSignature: Ubuntu 5.4.0-81.91-generic 5.4.128
RelatedPackageVersions:
 linux-restricted-modules-5.4.0-81-generic N/A
 linux-backports-modules-5.4.0-81-generic N/A
 linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: focal uec-images
Uname: Linux 5.4.0-81-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: 1.10.2-1ubuntu1
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-4.2
dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.2-1ubuntu1:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr21.2.0:cvnQEMU:ct1:cvrpc-i440fx-4.2:
dmi.product.family: Virtual Machine
dmi.product.name: OpenStack Nova
dmi.product.version: 21.2.0
dmi.sys.vendor: OpenStack Foundation

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1940928

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
tags: added: focal
Revision history for this message
Diko Parvanov (dparv) wrote : CurrentDmesg.txt

apport information

tags: added: apport-collected uec-images
description: updated
Revision history for this message
Diko Parvanov (dparv) wrote : Lspci.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : Lspci-vt.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : Lsusb-v.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : ProcModules.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : UdevDb.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : WifiSyslog.txt

apport information

Revision history for this message
Diko Parvanov (dparv) wrote : acpidump.txt

apport information

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.