libvirt CPU info doesn't count NUMA cells

Fix proposed to branch: master
Review: https://review.openstack.org/24631

Please clarify exactly what functionality is broken with the current code. AFAICT there is nothing in Nova which deals with NUMA info, even if the libvirt driver were to report it. Without knowing what is broken, it is hard to know if the proposed change is the right fix.

Patch Set 1:

I suppose the bug really is around how sockets are being reported on NUMA based systems, not necessarily with reporting on the NUMA cell count.

Most of this is copied from a comment I made on the review but cross posting here for those who aren't looking at the review.

On a dual socket Intel Xeon E5-2650 system the topology being reported via nova is the following.

"topology": {"cores": "8", "threads": "2", "sockets": "1"}

This is accurate in that it is what is reported via the getCapabilities call, but it is not accurate from the perspective of what is physically present on the system. So I believe we still need to fix this in some way to properly report on NUMA systems. 3rd party software that is trying to rely on the API can't get an accurate representation of what is physically present today.

I assume (and Daniel please correct me if I am wrong) that libvirt reports that based upon the understanding that this is per NUMA cell? Or is this a case were we need a newer version of libvirt that properly reports on a NUMA system? We don't have any asymmetric NUMA systems to look at so I don't know how its presented in the asymmetric case. What would be the correct way of reporting the number of cores, threads, and sockets on a NUMA based system, without representing the entire NUMA topology?

Cross-posted from the review:

If you're saying that the only acceptable solution is to expose the NUMA topology, then that is fine, we'll draft a blueprint and pitch it for Havana.

But at a minimum, the fix for this bug in Grizzly should be an entry in the Errata stating that CPU counts out of the Nova API are going inaccurate on NUMA nodes.

Looks like this is better served as part of this blueprint:

https://blueprints.launchpad.net/nova/+spec/nova-virt-numa-and-vcpu-topology

I think the answer to this problem really depends on what the user of the this data is expecting to get

If the user is taking the toplogy data and doing cpus = cores * sockets * threads, then merely adding in a numa node count is still going to give them broken information, since you can't assume NUMA topologies are symetrical.

So if users need a total CPU count for the host, then we should really just expose a total CPU number explicitly, and not require them to multiply together cores/sockets/threads.

If users actually want to see the NUMA topology, then we have a much bigger job than just adding numa cell count.

The patch was abandoned over a year ago and given the possibly related blueprints, marking this as incomplete given the confusion of what's at issue here.

this is really old, and numa support has evolved a ton since then. Marking as invalid. Please reopen if still happens in master.

Fix proposed to branch: master
Review: https://review.openstack.org/223869

https://review.openstack.org/223869

Reviving this old issue since trying to generate reports about hypervisor topology using statistics from novaclient yields some hilariously wrong results against NUMA-enabled compute nodes.

We're fans of this approach; let n-cpu count up the cells/sockets/cores/threads in the system and return them as cpu_info attributes. This way the consuming applications can just use the totals given, and don't need to make any assumptions that could be of dubious nature on exotic hardware profiles.

Reviewed: https://review.openstack.org/223869
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9aec873f75389c676af1b87f6455bf95a6813290
Submitter: Jenkins
Branch: master

commit 9aec873f75389c676af1b87f6455bf95a6813290
Author: Nicolas Simonds <email address hidden>
Date: Mon Sep 14 18:57:46 2015 -0700

    libvirt: Add NUMA cell count to cpu_info

    Adds the number of NUMA cells from libvirt to the cpu_info['topology']
    attribute, so that clients using that data for socket/core/thread
    counting on hypervisors can get better-quality results.

    Closes-Bug: 1156456
    Change-Id: I945f32bce96f27939d39b86af6ccec73b754f886