RPC endpoints take a long time to come up and accept connections

Bug #1925365 reported by Björn Tillenius
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
High
Alberto Donato

Bug Description

This is with 3.0.0~beta4 (9892-g.1c7495a4a)

On our internal MAAS, labmaas, it takes a very long time
for rackd to connect to regiond.

It takes about 90 seconds for regiond to advertise its RPC
endpoints at:

  http://10.245.136.7:5240/MAAS/rpc/

After that, the logs shows that rackd connects, but again,
it takes a very long time until regiond finally sets
rackd to be connected. So all in all, it takes several
minutes for rackd to finally be fully connected to regiond.

With beta3, the whole process takes a couple of seconds.

Changed in maas:
status: New → Triaged
importance: Undecided → High
milestone: none → 3.0.0-beta4
description: updated
Revision history for this message
Björn Tillenius (bjornt) wrote :

I tried to install different versions of the snap on labmaas, and all of them had the same problem, except for beta3. Even the snap with commit 50b84b60f57bf38fc9ad5d9c8c24bb45c893e11f in it. That is the commit that is directly after the beta3 release.

That commit changes to build the resource binary with the deb based go, instead of using the snapped (and newer version).

If I try to execute the resource binary from within the span confinement, it does indeed stall for quite a while, before eventually finishing. If I run the resource binary outside of the snap confinement, it finishes quickly. This is with commit 50b84b60f57bf38fc9ad5d9c8c24bb45c893e11f

With beta3, the resource binary takes a second or so to finish executing, both within and outside snap confinement.

Revision history for this message
Björn Tillenius (bjornt) wrote :

Looking at the strace, it tries to connect to 2a00:da80:fff0:2::2 and 46.255.230.98.

It looks like that's http://pci-ids.ucw.cz/.

From outside the snap confinement that doesn't happen, which probably means that it can't find /usr/share/misc/pci.ids that's in the snap.

That file is in the same location in both of the snaps, both beta3 and beta4, and it the same content in both of the snaps.

Changed in maas:
status: Triaged → Fix Committed
assignee: nobody → Alberto Donato (ack)
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.