Rack keeps opening idle RPC connections over the limits
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
MAAS | Status tracked in 3.5 | |||||
3.3 |
New
|
Undecided
|
Unassigned | |||
3.4 |
Triaged
|
Undecided
|
Unassigned | |||
3.5 |
Triaged
|
Undecided
|
Unassigned |
Bug Description
As per title, the rack is opening idle connections without limits under no load.
Steps to reproduce:
1) add a log statement like
```
```
in provisioningser
2) install maas from sources
3) maas init region+rack
4) wait **some minutes** and take a look at the logs:
```
Aug 24 12:35:19 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:35:20 maas-edge maas-rackd[10519]: provisioningser
...
CONNECTIONS!
Aug 24 12:35:45 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:35:46 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:35:47 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:35:48 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:35:49 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:36:19 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:36:49 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:37:19 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:37:49 maas-edge maas-rackd[10519]: provisioningser
Aug 24 12:38:19 maas-edge maas-rackd[10519]: provisioningser
```
and they keep increasing every 30 seconds. In addition to this strange behaviour, it is actually exceeding the configuration limits which are
```
max_
"The nominal number of connections to have per endpoint",
)
max_
"The maximum number of connections to scale to when under load",
)
```
If the region has 4 workers, it reaches 20 connections and it stops. If I specify the max max_rpc_
Note that they are all open connections. As a matter of facts, in the regiond logs there are no "disconnect" messages and if you kill the rackd process you will see as many disconnect messages as the number of open connections in the rack
summary: |
- Rack keeps opening idle connections over the limits + Rack keeps opening idle RPC connections over the limits |
description: | updated |
description: | updated |
description: | updated |
description: | updated |