boot-resources read is slow when there are a lot of images that have been deployed a lot of times
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Fix Committed
|
High
|
Stamatis Katsaounis | ||
3.4 |
In Progress
|
High
|
Stamatis Katsaounis | ||
3.5 |
Fix Released
|
High
|
Stamatis Katsaounis | ||
3.6 |
In Progress
|
High
|
Stamatis Katsaounis |
Bug Description
Describe the bug:
This is the same as bug 2091001, except that instead of taking 40 seconds
to list the images, it now takes around 16 seconds, which is still too much.
Steps to reproduce:
See bug 2091001
Expected behavior (what should have happened?):
boot-resources read should take a second or so to execute.
Actual behavior (what actually happened?):
boot-resources read takes 16 seconds.
MAAS version and installation type (deb, snap):
3.5.3, snap.
MAAS setup (HA, single node, multiple regions/racks):
3 nodes, region+rack.
Host OS distro and version:
Ubuntu 22.04.
Additional context:
Today, we loop over each image and get the last deploy time by searching through
the events table.
The proper fix would be bug 2095476, but in order to make 3.5 and 3.6 faster, we
can improve how we search the events table.
Instead of issuing one query per image, let's instead get the relevant events from
the database in one query and process the in Python instead.
maas_region_db=> select count(*), description from maasserver_event where type_id=42 group by description;
count | description
-------
144 | deployed ubuntu/
4706 | deployed ubuntu/
13 | deployed ubuntu/
1613 | deployed ubuntu/
216 | deployed ubuntu/
15 | deployed ubuntu/
993 | deployed ubuntu/
681 | deployed ubuntu/
15 | deployed ubuntu/
1427 | deployed ubuntu/
571 | deployed ubuntu/
724 | deployed ubuntu/
16279 | deployed ubuntu/
493 | deployed ubuntu/
5395 | deployed ubuntu/
9 | deployed ubuntu/
(16 rows)
maas_region_db=> explain analyze select count(*), description from maasserver_event where type_id=42 group by description;
maas_region_db=> explain analyze select count(*), description from maasserver_event where type_id=42 group by description;
-------
HashAggregate (cost=83096.
Group Key: description
Batches: 1 Memory Usage: 49kB
-> Bitmap Heap Scan on maasserver_event (cost=418.
Recheck Cond: (type_id = 42)
Heap Blocks: exact=29656
-> Bitmap Index Scan on maasserver_
Planning Time: 0.158 ms
Execution Time: 61.618 ms
(10 rows)
Related branches
- MAAS Lander: Needs Fixing
- MAAS Maintainers: Pending requested
-
Diff: 376 lines (+175/-26)6 files modifiedsrc/maasserver/api/boot_resources.py (+23/-6)
src/maasserver/api/tests/test_boot_resources.py (+1/-0)
src/maasserver/models/bootresource.py (+35/-1)
src/maasserver/models/node.py (+4/-0)
src/maasserver/models/tests/test_bootresource.py (+105/-1)
src/maasserver/websockets/handlers/bootresource.py (+7/-18)
- MAAS Lander: Approve
- Stamatis Katsaounis: Approve
-
Diff: 61 lines (+6/-12)1 file modifiedsrc/maasserver/websockets/handlers/bootresource.py (+6/-12)
- MAAS Lander: Approve
- Jacopo Rota: Approve
-
Diff: 61 lines (+6/-12)1 file modifiedsrc/maasserver/websockets/handlers/bootresource.py (+6/-12)
- MAAS Lander: Needs Fixing
- Jacopo Rota: Needs Fixing
-
Diff: 408 lines (+147/-61)6 files modifiedsrc/maasserver/api/boot_resources.py (+23/-6)
src/maasserver/api/tests/test_boot_resources.py (+1/-3)
src/maasserver/models/bootresource.py (+34/-20)
src/maasserver/models/node.py (+4/-0)
src/maasserver/models/tests/test_bootresource.py (+78/-14)
src/maasserver/websockets/handlers/bootresource.py (+7/-18)
- MAAS Lander: Approve
- Jacopo Rota: Approve
-
Diff: 390 lines (+150/-58)6 files modifiedsrc/maasserver/api/boot_resources.py (+23/-6)
src/maasserver/api/tests/test_boot_resources.py (+1/-3)
src/maasserver/models/bootresource.py (+34/-20)
src/maasserver/models/node.py (+4/-0)
src/maasserver/models/tests/test_bootresource.py (+78/-14)
src/maasserver/websockets/handlers/bootresource.py (+10/-15)
- MAAS Lander: Approve
- Björn Tillenius: Approve
-
Diff: 390 lines (+150/-58)6 files modifiedsrc/maasserver/api/boot_resources.py (+23/-6)
src/maasserver/api/tests/test_boot_resources.py (+1/-3)
src/maasserver/models/bootresource.py (+34/-20)
src/maasserver/models/node.py (+4/-0)
src/maasserver/models/tests/test_bootresource.py (+78/-14)
src/maasserver/websockets/handlers/bootresource.py (+10/-15)
Changed in maas: | |
status: | New → Triaged |
importance: | Undecided → High |
milestone: | none → 3.7.0 |
Changed in maas: | |
assignee: | nobody → Stamatis Katsaounis (skatsaounis) |
status: | Triaged → In Progress |
Changed in maas: | |
status: | In Progress → Fix Committed |
Changed in maas: | |
status: | Fix Committed → In Progress |
Changed in maas: | |
status: | In Progress → Fix Committed |
tags: | added: bug-council |