Too many metadata agents on compute node in DVR
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Mirantis OpenStack | Status tracked in 10.0.x | |||||
10.0.x |
Fix Committed
|
High
|
Sergey Kolekonov | |||
7.0.x |
Won't Fix
|
High
|
MOS Maintenance | |||
8.0.x |
Won't Fix
|
High
|
MOS Maintenance | |||
9.x |
Fix Released
|
High
|
Sergey Kolekonov |
Bug Description
Detailed bug description:
There are N (N=cores) metadata-agent processes on compute node. Each process holds connection to RabbitMQ, resulting in N*computes of total connections (almost 10k at 200 nodes with 48 cores)
Steps to reproduce:
Deploy MOS with Neutron DVR. Boot instances at the compute node (see comment #3 for reasoning)
Expected results:
I'd expect less number of metadata agent process since having so many of them looks like overkill.
Actual result:
One process per core.
Reproducibility:
just deploy
Workaround:
no
Impact:
too many connections to RabbitMQ in large scale
Description of the environment:
- Operation system: Ubuntu
- Versions of components: MOS 9.0
- Reference architecture: MOS classic, Neutron DVR
- Network model: VxLAN
- Related projects installed: none
root@node-2:~# lsof -i 4 | grep 5673 | grep neutron-m node-2. domain. tld:58859- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:33659- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:33796- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:48403- >messaging- node-138. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:59400- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:51428- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:59454- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:33744- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:51398- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:39521- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:48406- >messaging- node-138. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:59729- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:59730- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:48412- >messaging- node-138. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:39476- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:40052- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:39466- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:39467- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:59391- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:59392- >messaging- node-140. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:48404- >messaging- node-138. domain. tld:5673 (ESTABLISHED) node-2. domain. tld:43495- >messa. ..
neutron-m 48784 neutron 10u IPv4 189433 0t0 TCP messaging-
neutron-m 48784 neutron 13u IPv4 13134201 0t0 TCP messaging-
neutron-m 48795 neutron 10u IPv4 12755678 0t0 TCP messaging-
neutron-m 48797 neutron 10u IPv4 13024039 0t0 TCP messaging-
neutron-m 48797 neutron 12u IPv4 1328637 0t0 TCP messaging-
neutron-m 48798 neutron 10u IPv4 12610008 0t0 TCP messaging-
neutron-m 48798 neutron 12u IPv4 1315489 0t0 TCP messaging-
neutron-m 48799 neutron 10u IPv4 13024033 0t0 TCP messaging-
neutron-m 48800 neutron 10u IPv4 12607908 0t0 TCP messaging-
neutron-m 48800 neutron 12u IPv4 1791292 0t0 TCP messaging-
neutron-m 48801 neutron 10u IPv4 12904059 0t0 TCP messaging-
neutron-m 48802 neutron 11u IPv4 3129010 0t0 TCP messaging-
neutron-m 48802 neutron 12u IPv4 3126438 0t0 TCP messaging-
neutron-m 48803 neutron 10u IPv4 13118538 0t0 TCP messaging-
neutron-m 48803 neutron 12u IPv4 1724330 0t0 TCP messaging-
neutron-m 48805 neutron 10u IPv4 12904111 0t0 TCP messaging-
neutron-m 48805 neutron 11u IPv4 1763638 0t0 TCP messaging-
neutron-m 48805 neutron 12u IPv4 1790597 0t0 TCP messaging-
neutron-m 48807 neutron 11u IPv4 949184 0t0 TCP messaging-
neutron-m 48807 neutron 12u IPv4 1332763 0t0 TCP messaging-
neutron-m 48808 neutron 10u IPv4 12904058 0t0 TCP messaging-
neutron-m 48808 neutron 12u IPv4 2038553 0t0 TCP messaging-