:~$ virsh nodeinfo
CPU model: x86_64
CPU(s): 64
CPU frequency: 1400 MHz
CPU socket(s): 1
Core(s) per socket: 64
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 528376144 KiB
but it should be "NUMA cell(s): 8".
I'm running a 3.13.0-48-generic equipped with _four_ physical AMD Opteron 6282 SE to a Supermicro H8QG6-F. libvirt-bin version: 1.2.2-0ubuntu13.1.10
As well as lstopo is resulting in eight NUMA cells (see attached picture).
But "virsh nodeinfo" shows only only NUMA cell with one socket with 64 CPUs (single threaded). Performance of virtualized machines one this host is very bad.
I think it has something to do with the identical core_ids per physical socket and module.
:~$ cat /proc/cpuinfo | egrep "processor|physical id|siblings|core id|cpu cores"
processor : 0
physical id : 0
siblings : 16
core id : 0
cpu cores : 8
processor : 1
physical id : 0
siblings : 16
core id : 1
cpu cores : 8
processor : 2
physical id : 0
siblings : 16
core id : 2
cpu cores : 8
processor : 3
physical id : 0
siblings : 16
core id : 3
cpu cores : 8
processor : 4
physical id : 0
siblings : 16
core id : 4
cpu cores : 8
processor : 5
physical id : 0
siblings : 16
core id : 5
cpu cores : 8
processor : 6
physical id : 0
siblings : 16
core id : 6
cpu cores : 8
processor : 7
physical id : 0
siblings : 16
core id : 7
cpu cores : 8
processor : 8
physical id : 0
siblings : 16
core id : 0
cpu cores : 8
processor : 9
physical id : 0
siblings : 16
core id : 1
cpu cores : 8
processor : 10
physical id : 0
siblings : 16
core id : 2
cpu cores : 8
processor : 11
physical id : 0
siblings : 16
core id : 3
cpu cores : 8
processor : 12
physical id : 0
siblings : 16
core id : 4
cpu cores : 8
processor : 13
physical id : 0
siblings : 16
core id : 5
cpu cores : 8
processor : 14
physical id : 0
siblings : 16
core id : 6
cpu cores : 8
processor : 15
physical id : 0
siblings : 16
core id : 7
cpu cores : 8
processor : 16
physical id : 1
siblings : 16
core id : 0
cpu cores : 8
processor : 17
physical id : 1
siblings : 16
core id : 1
cpu cores : 8
processor : 18
physical id : 1
siblings : 16
core id : 2
cpu cores : 8
processor : 19
physical id : 1
siblings : 16
core id : 3
cpu cores : 8
processor : 20
physical id : 1
siblings : 16
core id : 4
cpu cores : 8
processor : 21
physical id : 1
siblings : 16
core id : 5
cpu cores : 8
processor : 22
physical id : 1
siblings : 16
core id : 6
cpu cores : 8
processor : 23
physical id : 1
siblings : 16
core id : 7
cpu cores : 8
processor : 24
physical id : 1
siblings : 16
core id : 0
cpu cores : 8
processor : 25
physical id : 1
siblings : 16
core id : 1
cpu cores : 8
processor : 26
physical id : 1
siblings : 16
core id : 2
cpu cores : 8
processor : 27
physical id : 1
siblings : 16
core id : 3
cpu cores : 8
processor : 28
physical id : 1
siblings : 16
core id : 4
cpu cores : 8
processor : 29
physical id : 1
siblings : 16
core id : 5
cpu cores : 8
processor : 30
physical id : 1
siblings : 16
core id : 6
cpu cores : 8
processor : 31
physical id : 1
siblings : 16
core id : 7
cpu cores : 8
processor : 32
physical id : 2
siblings : 16
core id : 0
cpu cores : 8
processor : 33
physical id : 2
siblings : 16
core id : 1
cpu cores : 8
processor : 34
physical id : 2
siblings : 16
core id : 2
cpu cores : 8
processor : 35
physical id : 2
siblings : 16
core id : 3
cpu cores : 8
processor : 36
physical id : 2
siblings : 16
core id : 4
cpu cores : 8
processor : 37
physical id : 2
siblings : 16
core id : 5
cpu cores : 8
processor : 38
physical id : 2
siblings : 16
core id : 6
cpu cores : 8
processor : 39
physical id : 2
siblings : 16
core id : 7
cpu cores : 8
processor : 40
physical id : 2
siblings : 16
core id : 0
cpu cores : 8
processor : 41
physical id : 2
siblings : 16
core id : 1
cpu cores : 8
processor : 42
physical id : 2
siblings : 16
core id : 2
cpu cores : 8
processor : 43
physical id : 2
siblings : 16
core id : 3
cpu cores : 8
processor : 44
physical id : 2
siblings : 16
core id : 4
cpu cores : 8
processor : 45
physical id : 2
siblings : 16
core id : 5
cpu cores : 8
processor : 46
physical id : 2
siblings : 16
core id : 6
cpu cores : 8
processor : 47
physical id : 2
siblings : 16
core id : 7
cpu cores : 8
processor : 48
physical id : 3
siblings : 16
core id : 0
cpu cores : 8
processor : 49
physical id : 3
siblings : 16
core id : 1
cpu cores : 8
processor : 50
physical id : 3
siblings : 16
core id : 2
cpu cores : 8
processor : 51
physical id : 3
siblings : 16
core id : 3
cpu cores : 8
processor : 52
physical id : 3
siblings : 16
core id : 4
cpu cores : 8
processor : 53
physical id : 3
siblings : 16
core id : 5
cpu cores : 8
processor : 54
physical id : 3
siblings : 16
core id : 6
cpu cores : 8
processor : 55
physical id : 3
siblings : 16
core id : 7
cpu cores : 8
processor : 56
physical id : 3
siblings : 16
core id : 0
cpu cores : 8
processor : 57
physical id : 3
siblings : 16
core id : 1
cpu cores : 8
processor : 58
physical id : 3
siblings : 16
core id : 2
cpu cores : 8
processor : 59
physical id : 3
siblings : 16
core id : 3
cpu cores : 8
processor : 60
physical id : 3
siblings : 16
core id : 4
cpu cores : 8
processor : 61
physical id : 3
siblings : 16
core id : 5
cpu cores : 8
processor : 62
physical id : 3
siblings : 16
core id : 6
cpu cores : 8
processor : 63
physical id : 3
siblings : 16
core id : 7
cpu cores : 8
1)
Description: Ubuntu 14.04.2 LTS
Release: 14.0
2) .1.10 .1.10 .1.10 0 archive. ubuntu. com/ubuntu/ trusty-updates/main amd64 Packages dpkg/status 2.2-0ubuntu13. 1.7 0 security. ubuntu. com/ubuntu/ trusty- security/ main amd64 Packages 2.2-0ubuntu13 0 archive. ubuntu. com/ubuntu/ trusty/main amd64 Packages
libvirt-bin:
Installed: 1.2.2-0ubuntu13
Candidate: 1.2.2-0ubuntu13
Version table:
*** 1.2.2-0ubuntu13
500 http://
100 /var/lib/
1.
500 http://
1.
500 http://
4+5)
"virsh nodeinfo" shows wrong NUMA topology:
:~$ virsh nodeinfo
CPU model: x86_64
CPU(s): 64
CPU frequency: 1400 MHz
CPU socket(s): 1
Core(s) per socket: 64
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 528376144 KiB
but it should be "NUMA cell(s): 8".
I'm running a 3.13.0-48-generic equipped with _four_ physical AMD Opteron 6282 SE to a Supermicro H8QG6-F. libvirt-bin version: 1.2.2-0ubuntu13 .1.10
NUMA ctl gives me eight cells:
:~$ numactl -H
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3 4 5 6 7
node 0 size: 64431 MB
node 0 free: 58505 MB
node 1 cpus: 8 9 10 11 12 13 14 15
node 1 size: 64510 MB
node 1 free: 45068 MB
node 2 cpus: 16 17 18 19 20 21 22 23
node 2 size: 64510 MB
node 2 free: 62910 MB
node 3 cpus: 24 25 26 27 28 29 30 31
node 3 size: 64510 MB
node 3 free: 20460 MB
node 4 cpus: 32 33 34 35 36 37 38 39
node 4 size: 64510 MB
node 4 free: 63512 MB
node 5 cpus: 40 41 42 43 44 45 46 47
node 5 size: 64510 MB
node 5 free: 52723 MB
node 6 cpus: 48 49 50 51 52 53 54 55
node 6 size: 64510 MB
node 6 free: 64084 MB
node 7 cpus: 56 57 58 59 60 61 62 63
node 7 size: 64494 MB
node 7 free: 62013 MB
node distances:
node 0 1 2 3 4 5 6 7
0: 10 16 16 22 16 22 16 22
1: 16 10 22 16 22 16 22 16
2: 16 22 10 16 16 22 16 22
3: 22 16 16 10 22 16 22 16
4: 16 22 16 22 10 16 16 22
5: 22 16 22 16 16 10 22 16
6: 16 22 16 22 16 22 10 16
7: 22 16 22 16 22 16 16 10
As well as lstopo is resulting in eight NUMA cells (see attached picture).
But "virsh nodeinfo" shows only only NUMA cell with one socket with 64 CPUs (single threaded). Performance of virtualized machines one this host is very bad.
I think it has something to do with the identical core_ids per physical socket and module.
:~$ cat /proc/cpuinfo | egrep "processor|physical id|siblings|core id|cpu cores"
processor : 0
physical id : 0
siblings : 16
core id : 0
cpu cores : 8
processor : 1
physical id : 0
siblings : 16
core id : 1
cpu cores : 8
processor : 2
physical id : 0
siblings : 16
core id : 2
cpu cores : 8
processor : 3
physical id : 0
siblings : 16
core id : 3
cpu cores : 8
processor : 4
physical id : 0
siblings : 16
core id : 4
cpu cores : 8
processor : 5
physical id : 0
siblings : 16
core id : 5
cpu cores : 8
processor : 6
physical id : 0
siblings : 16
core id : 6
cpu cores : 8
processor : 7
physical id : 0
siblings : 16
core id : 7
cpu cores : 8
processor : 8
physical id : 0
siblings : 16
core id : 0
cpu cores : 8
processor : 9
physical id : 0
siblings : 16
core id : 1
cpu cores : 8
processor : 10
physical id : 0
siblings : 16
core id : 2
cpu cores : 8
processor : 11
physical id : 0
siblings : 16
core id : 3
cpu cores : 8
processor : 12
physical id : 0
siblings : 16
core id : 4
cpu cores : 8
processor : 13
physical id : 0
siblings : 16
core id : 5
cpu cores : 8
processor : 14
physical id : 0
siblings : 16
core id : 6
cpu cores : 8
processor : 15
physical id : 0
siblings : 16
core id : 7
cpu cores : 8
processor : 16
physical id : 1
siblings : 16
core id : 0
cpu cores : 8
processor : 17
physical id : 1
siblings : 16
core id : 1
cpu cores : 8
processor : 18
physical id : 1
siblings : 16
core id : 2
cpu cores : 8
processor : 19
physical id : 1
siblings : 16
core id : 3
cpu cores : 8
processor : 20
physical id : 1
siblings : 16
core id : 4
cpu cores : 8
processor : 21
physical id : 1
siblings : 16
core id : 5
cpu cores : 8
processor : 22
physical id : 1
siblings : 16
core id : 6
cpu cores : 8
processor : 23
physical id : 1
siblings : 16
core id : 7
cpu cores : 8
processor : 24
physical id : 1
siblings : 16
core id : 0
cpu cores : 8
processor : 25
physical id : 1
siblings : 16
core id : 1
cpu cores : 8
processor : 26
physical id : 1
siblings : 16
core id : 2
cpu cores : 8
processor : 27
physical id : 1
siblings : 16
core id : 3
cpu cores : 8
processor : 28
physical id : 1
siblings : 16
core id : 4
cpu cores : 8
processor : 29
physical id : 1
siblings : 16
core id : 5
cpu cores : 8
processor : 30
physical id : 1
siblings : 16
core id : 6
cpu cores : 8
processor : 31
physical id : 1
siblings : 16
core id : 7
cpu cores : 8
processor : 32
physical id : 2
siblings : 16
core id : 0
cpu cores : 8
processor : 33
physical id : 2
siblings : 16
core id : 1
cpu cores : 8
processor : 34
physical id : 2
siblings : 16
core id : 2
cpu cores : 8
processor : 35
physical id : 2
siblings : 16
core id : 3
cpu cores : 8
processor : 36
physical id : 2
siblings : 16
core id : 4
cpu cores : 8
processor : 37
physical id : 2
siblings : 16
core id : 5
cpu cores : 8
processor : 38
physical id : 2
siblings : 16
core id : 6
cpu cores : 8
processor : 39
physical id : 2
siblings : 16
core id : 7
cpu cores : 8
processor : 40
physical id : 2
siblings : 16
core id : 0
cpu cores : 8
processor : 41
physical id : 2
siblings : 16
core id : 1
cpu cores : 8
processor : 42
physical id : 2
siblings : 16
core id : 2
cpu cores : 8
processor : 43
physical id : 2
siblings : 16
core id : 3
cpu cores : 8
processor : 44
physical id : 2
siblings : 16
core id : 4
cpu cores : 8
processor : 45
physical id : 2
siblings : 16
core id : 5
cpu cores : 8
processor : 46
physical id : 2
siblings : 16
core id : 6
cpu cores : 8
processor : 47
physical id : 2
siblings : 16
core id : 7
cpu cores : 8
processor : 48
physical id : 3
siblings : 16
core id : 0
cpu cores : 8
processor : 49
physical id : 3
siblings : 16
core id : 1
cpu cores : 8
processor : 50
physical id : 3
siblings : 16
core id : 2
cpu cores : 8
processor : 51
physical id : 3
siblings : 16
core id : 3
cpu cores : 8
processor : 52
physical id : 3
siblings : 16
core id : 4
cpu cores : 8
processor : 53
physical id : 3
siblings : 16
core id : 5
cpu cores : 8
processor : 54
physical id : 3
siblings : 16
core id : 6
cpu cores : 8
processor : 55
physical id : 3
siblings : 16
core id : 7
cpu cores : 8
processor : 56
physical id : 3
siblings : 16
core id : 0
cpu cores : 8
processor : 57
physical id : 3
siblings : 16
core id : 1
cpu cores : 8
processor : 58
physical id : 3
siblings : 16
core id : 2
cpu cores : 8
processor : 59
physical id : 3
siblings : 16
core id : 3
cpu cores : 8
processor : 60
physical id : 3
siblings : 16
core id : 4
cpu cores : 8
processor : 61
physical id : 3
siblings : 16
core id : 5
cpu cores : 8
processor : 62
physical id : 3
siblings : 16
core id : 6
cpu cores : 8
processor : 63
physical id : 3
siblings : 16
core id : 7
cpu cores : 8
Thanks!
ProblemType: Bug .1.10 ature: Ubuntu 3.13.0- 48.80-generic 3.13.11-ckt16
DistroRelease: Ubuntu 14.04
Package: libvirt-bin 1.2.2-0ubuntu13
ProcVersionSign
Uname: Linux 3.13.0-48-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.10
Architecture: amd64
Date: Mon Apr 20 13:04:31 2015
InstallationDate: Installed on 2012-12-07 (864 days ago)
InstallationMedia: Ubuntu-Server 12.04.1 LTS "Precise Pangolin" - Release amd64 (20120817.3)
KernLog:
SourcePackage: libvirt conffile. .etc.libvirt. qemu.conf: [inaccessible: [Errno 13] Permission denied: '/etc/libvirt/ qemu.conf' ]
UpgradeStatus: Upgraded to trusty on 2014-07-07 (286 days ago)
modified.