Comment 0 for bug 1446177

Revision history for this message
FliTTi (m-flittner) wrote :

1)
Description: Ubuntu 14.04.2 LTS
Release: 14.0

2)
libvirt-bin:
  Installed: 1.2.2-0ubuntu13.1.10
  Candidate: 1.2.2-0ubuntu13.1.10
  Version table:
 *** 1.2.2-0ubuntu13.1.10 0
        500 http://archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.2.2-0ubuntu13.1.7 0
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
     1.2.2-0ubuntu13 0
        500 http://archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

4+5)
"virsh nodeinfo" shows wrong NUMA topology:

:~$ virsh nodeinfo
CPU model: x86_64
CPU(s): 64
CPU frequency: 1400 MHz
CPU socket(s): 1
Core(s) per socket: 64
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 528376144 KiB

but it should be "NUMA cell(s): 8".

I'm running a 3.13.0-48-generic equipped with _four_ physical AMD Opteron 6282 SE to a Supermicro H8QG6-F. libvirt-bin version: 1.2.2-0ubuntu13.1.10

NUMA ctl gives me eight cells:
:~$ numactl -H
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3 4 5 6 7
node 0 size: 64431 MB
node 0 free: 58505 MB
node 1 cpus: 8 9 10 11 12 13 14 15
node 1 size: 64510 MB
node 1 free: 45068 MB
node 2 cpus: 16 17 18 19 20 21 22 23
node 2 size: 64510 MB
node 2 free: 62910 MB
node 3 cpus: 24 25 26 27 28 29 30 31
node 3 size: 64510 MB
node 3 free: 20460 MB
node 4 cpus: 32 33 34 35 36 37 38 39
node 4 size: 64510 MB
node 4 free: 63512 MB
node 5 cpus: 40 41 42 43 44 45 46 47
node 5 size: 64510 MB
node 5 free: 52723 MB
node 6 cpus: 48 49 50 51 52 53 54 55
node 6 size: 64510 MB
node 6 free: 64084 MB
node 7 cpus: 56 57 58 59 60 61 62 63
node 7 size: 64494 MB
node 7 free: 62013 MB
node distances:
node 0 1 2 3 4 5 6 7
  0: 10 16 16 22 16 22 16 22
  1: 16 10 22 16 22 16 22 16
  2: 16 22 10 16 16 22 16 22
  3: 22 16 16 10 22 16 22 16
  4: 16 22 16 22 10 16 16 22
  5: 22 16 22 16 16 10 22 16
  6: 16 22 16 22 16 22 10 16
  7: 22 16 22 16 22 16 16 10

As well as lstopo is resulting in eight NUMA cells (see attached picture).

But "virsh nodeinfo" shows only only NUMA cell with one socket with 64 CPUs (single threaded). Performance of virtualized machines one this host is very bad.

I think it has something to do with the identical core_ids per physical socket and module.

:~$ cat /proc/cpuinfo | egrep "processor|physical id|siblings|core id|cpu cores"
processor : 0
physical id : 0
siblings : 16
core id : 0
cpu cores : 8
processor : 1
physical id : 0
siblings : 16
core id : 1
cpu cores : 8
processor : 2
physical id : 0
siblings : 16
core id : 2
cpu cores : 8
processor : 3
physical id : 0
siblings : 16
core id : 3
cpu cores : 8
processor : 4
physical id : 0
siblings : 16
core id : 4
cpu cores : 8
processor : 5
physical id : 0
siblings : 16
core id : 5
cpu cores : 8
processor : 6
physical id : 0
siblings : 16
core id : 6
cpu cores : 8
processor : 7
physical id : 0
siblings : 16
core id : 7
cpu cores : 8
processor : 8
physical id : 0
siblings : 16
core id : 0
cpu cores : 8
processor : 9
physical id : 0
siblings : 16
core id : 1
cpu cores : 8
processor : 10
physical id : 0
siblings : 16
core id : 2
cpu cores : 8
processor : 11
physical id : 0
siblings : 16
core id : 3
cpu cores : 8
processor : 12
physical id : 0
siblings : 16
core id : 4
cpu cores : 8
processor : 13
physical id : 0
siblings : 16
core id : 5
cpu cores : 8
processor : 14
physical id : 0
siblings : 16
core id : 6
cpu cores : 8
processor : 15
physical id : 0
siblings : 16
core id : 7
cpu cores : 8
processor : 16
physical id : 1
siblings : 16
core id : 0
cpu cores : 8
processor : 17
physical id : 1
siblings : 16
core id : 1
cpu cores : 8
processor : 18
physical id : 1
siblings : 16
core id : 2
cpu cores : 8
processor : 19
physical id : 1
siblings : 16
core id : 3
cpu cores : 8
processor : 20
physical id : 1
siblings : 16
core id : 4
cpu cores : 8
processor : 21
physical id : 1
siblings : 16
core id : 5
cpu cores : 8
processor : 22
physical id : 1
siblings : 16
core id : 6
cpu cores : 8
processor : 23
physical id : 1
siblings : 16
core id : 7
cpu cores : 8
processor : 24
physical id : 1
siblings : 16
core id : 0
cpu cores : 8
processor : 25
physical id : 1
siblings : 16
core id : 1
cpu cores : 8
processor : 26
physical id : 1
siblings : 16
core id : 2
cpu cores : 8
processor : 27
physical id : 1
siblings : 16
core id : 3
cpu cores : 8
processor : 28
physical id : 1
siblings : 16
core id : 4
cpu cores : 8
processor : 29
physical id : 1
siblings : 16
core id : 5
cpu cores : 8
processor : 30
physical id : 1
siblings : 16
core id : 6
cpu cores : 8
processor : 31
physical id : 1
siblings : 16
core id : 7
cpu cores : 8
processor : 32
physical id : 2
siblings : 16
core id : 0
cpu cores : 8
processor : 33
physical id : 2
siblings : 16
core id : 1
cpu cores : 8
processor : 34
physical id : 2
siblings : 16
core id : 2
cpu cores : 8
processor : 35
physical id : 2
siblings : 16
core id : 3
cpu cores : 8
processor : 36
physical id : 2
siblings : 16
core id : 4
cpu cores : 8
processor : 37
physical id : 2
siblings : 16
core id : 5
cpu cores : 8
processor : 38
physical id : 2
siblings : 16
core id : 6
cpu cores : 8
processor : 39
physical id : 2
siblings : 16
core id : 7
cpu cores : 8
processor : 40
physical id : 2
siblings : 16
core id : 0
cpu cores : 8
processor : 41
physical id : 2
siblings : 16
core id : 1
cpu cores : 8
processor : 42
physical id : 2
siblings : 16
core id : 2
cpu cores : 8
processor : 43
physical id : 2
siblings : 16
core id : 3
cpu cores : 8
processor : 44
physical id : 2
siblings : 16
core id : 4
cpu cores : 8
processor : 45
physical id : 2
siblings : 16
core id : 5
cpu cores : 8
processor : 46
physical id : 2
siblings : 16
core id : 6
cpu cores : 8
processor : 47
physical id : 2
siblings : 16
core id : 7
cpu cores : 8
processor : 48
physical id : 3
siblings : 16
core id : 0
cpu cores : 8
processor : 49
physical id : 3
siblings : 16
core id : 1
cpu cores : 8
processor : 50
physical id : 3
siblings : 16
core id : 2
cpu cores : 8
processor : 51
physical id : 3
siblings : 16
core id : 3
cpu cores : 8
processor : 52
physical id : 3
siblings : 16
core id : 4
cpu cores : 8
processor : 53
physical id : 3
siblings : 16
core id : 5
cpu cores : 8
processor : 54
physical id : 3
siblings : 16
core id : 6
cpu cores : 8
processor : 55
physical id : 3
siblings : 16
core id : 7
cpu cores : 8
processor : 56
physical id : 3
siblings : 16
core id : 0
cpu cores : 8
processor : 57
physical id : 3
siblings : 16
core id : 1
cpu cores : 8
processor : 58
physical id : 3
siblings : 16
core id : 2
cpu cores : 8
processor : 59
physical id : 3
siblings : 16
core id : 3
cpu cores : 8
processor : 60
physical id : 3
siblings : 16
core id : 4
cpu cores : 8
processor : 61
physical id : 3
siblings : 16
core id : 5
cpu cores : 8
processor : 62
physical id : 3
siblings : 16
core id : 6
cpu cores : 8
processor : 63
physical id : 3
siblings : 16
core id : 7
cpu cores : 8

Thanks!

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: libvirt-bin 1.2.2-0ubuntu13.1.10
ProcVersionSignature: Ubuntu 3.13.0-48.80-generic 3.13.11-ckt16
Uname: Linux 3.13.0-48-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.10
Architecture: amd64
Date: Mon Apr 20 13:04:31 2015
InstallationDate: Installed on 2012-12-07 (864 days ago)
InstallationMedia: Ubuntu-Server 12.04.1 LTS "Precise Pangolin" - Release amd64 (20120817.3)
KernLog:

SourcePackage: libvirt
UpgradeStatus: Upgraded to trusty on 2014-07-07 (286 days ago)
modified.conffile..etc.libvirt.qemu.conf: [inaccessible: [Errno 13] Permission denied: '/etc/libvirt/qemu.conf']