| 2018-10-04 16:27:51 |
Scott Moser |
bug |
|
|
added bug |
| 2018-10-04 16:28:52 |
Scott Moser |
description |
The current squashfs/lxd image (20181002) is almost twice the size as the
previous image (20180928) and boots dramatically slower.
At first, just looking at manifests you'd blame systemd change, but I
think the real problem is seeded snaps.
$ lxc image list ubuntu-daily: | grep 18.10 | grep x86_64
| c (7 more) | 6cad1409aaac | yes | ubuntu 18.10 amd64 (daily) (20181002) | x86_64 | 323.85MB | Oct 2, 2018 at 12:00am (UTC) |
| | 21f94ea409ac | yes | ubuntu 18.10 amd64 (daily) (20180928) | x86_64 | 170.25MB | Sep 28, 2018 at 12:00am (UTC) |
| | 5011b07c621c | yes | ubuntu 18.10 amd64 (daily) (20180926) | x86_64 | 170.25MB | Sep 26, 2018 at 12:00am (UTC) |
## The old image
$ lxc launch ubuntu-daily:21f94ea409ac cosmic-test
$ lxc exec cosmic-test -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20180928
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/gyrNWkdSRG/
+ systemd-analyze
Startup finished in 11.822s (userspace)
graphical.target reached after 10.139s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @10.139s
└─multi-user.target @10.139s
└─snapd.seeded.service @6.041s +4.096s
└─snapd.service @5.770s +269ms
└─basic.target @5.698s
└─sockets.target @5.698s
└─snapd.socket @5.694s +3ms
└─sysinit.target @5.682s
└─cloud-init.service @4.676s +1.003s
└─systemd-networkd-wait-online.service @1.659s +3.015s
└─systemd-networkd.service @1.608s +49ms
└─network-pre.target @1.605s
└─cloud-init-local.service @198ms +1.406s
└─systemd-journald.socket @187ms
└─-.mount @184ms
└─system.slice @184ms
└─-.slice @184ms
## Try the current one.
$ lxc launch ubuntu-daily:cosmic c2
$ lxc exec c2 -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20181002
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/MCkrygb9sM/
+ systemd-analyze
Startup finished in 38.639s (userspace)
graphical.target reached after 36.990s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @36.990s
└─multi-user.target @36.989s
└─snapd.seeded.service @5.149s +31.839s
└─snapd.service @12.863s +67ms
└─snap-lxd-8959.mount @15.227s +15ms
└─snap.mount @11.543s +14ms
└─local-fs-pre.target @363ms
└─keyboard-setup.service @164ms +198ms
└─systemd-journald.socket @162ms
└─-.mount @159ms
└─system.slice @159ms
└─-.slice @159ms
Heres a diff of their manifests:
$ diff -u /tmp/20180928.manifest /tmp/20181002.manifest | grep "^[-+]"
--- /tmp/20180928.manifest 2018-10-04 12:16:28.593773558 -0400
+++ /tmp/20181002.manifest 2018-10-04 12:16:57.945328943 -0400
-apparmor 2.12-4ubuntu7
+apparmor 2.12-4ubuntu8
-libapparmor1:amd64 2.12-4ubuntu7
+libapparmor1:amd64 2.12-4ubuntu8
-libnss-systemd:amd64 239-7ubuntu8
+libnss-systemd:amd64 239-7ubuntu9
-libpam-systemd:amd64 239-7ubuntu8
+libpam-systemd:amd64 239-7ubuntu9
-libsystemd0:amd64 239-7ubuntu8
+libsystemd0:amd64 239-7ubuntu9
-libudev1:amd64 239-7ubuntu8
+libudev1:amd64 239-7ubuntu9
-libzstd1:amd64 1.3.3+dfsg-2ubuntu1
+libzstd1:amd64 1.3.5+dfsg-1ubuntu1
-networkd-dispatcher 1.7-0ubuntu6
+networkd-dispatcher 1.7-0ubuntu8
-python3-configobj 5.0.6-2
+python3-configobj 5.0.6-3
-shared-mime-info 1.9-2
+shared-mime-info 1.10-1
-systemd 239-7ubuntu8
-systemd-sysv 239-7ubuntu8
+systemd 239-7ubuntu9
+systemd-sysv 239-7ubuntu9
-udev 239-7ubuntu8
+udev 239-7ubuntu9 |
The current squashfs/lxd image (20181002) is almost twice the size
(170M -> 323M) as the previous image (20180928) and boots dramatically
slower.
At first, just looking at manifests you'd blame systemd change, but I
think the real problem is seeded snaps.
$ lxc image list ubuntu-daily: | grep 18.10 | grep x86_64
| c (7 more) | 6cad1409aaac | yes | ubuntu 18.10 amd64 (daily) (20181002) | x86_64 | 323.85MB | Oct 2, 2018 at 12:00am (UTC) |
| | 21f94ea409ac | yes | ubuntu 18.10 amd64 (daily) (20180928) | x86_64 | 170.25MB | Sep 28, 2018 at 12:00am (UTC) |
| | 5011b07c621c | yes | ubuntu 18.10 amd64 (daily) (20180926) | x86_64 | 170.25MB | Sep 26, 2018 at 12:00am (UTC) |
## The old image
$ lxc launch ubuntu-daily:21f94ea409ac cosmic-test
$ lxc exec cosmic-test -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20180928
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/gyrNWkdSRG/
+ systemd-analyze
Startup finished in 11.822s (userspace)
graphical.target reached after 10.139s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @10.139s
└─multi-user.target @10.139s
└─snapd.seeded.service @6.041s +4.096s
└─snapd.service @5.770s +269ms
└─basic.target @5.698s
└─sockets.target @5.698s
└─snapd.socket @5.694s +3ms
└─sysinit.target @5.682s
└─cloud-init.service @4.676s +1.003s
└─systemd-networkd-wait-online.service @1.659s +3.015s
└─systemd-networkd.service @1.608s +49ms
└─network-pre.target @1.605s
└─cloud-init-local.service @198ms +1.406s
└─systemd-journald.socket @187ms
└─-.mount @184ms
└─system.slice @184ms
└─-.slice @184ms
## Try the current one.
$ lxc launch ubuntu-daily:cosmic c2
$ lxc exec c2 -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20181002
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/MCkrygb9sM/
+ systemd-analyze
Startup finished in 38.639s (userspace)
graphical.target reached after 36.990s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @36.990s
└─multi-user.target @36.989s
└─snapd.seeded.service @5.149s +31.839s
└─snapd.service @12.863s +67ms
└─snap-lxd-8959.mount @15.227s +15ms
└─snap.mount @11.543s +14ms
└─local-fs-pre.target @363ms
└─keyboard-setup.service @164ms +198ms
└─systemd-journald.socket @162ms
└─-.mount @159ms
└─system.slice @159ms
└─-.slice @159ms
Heres a diff of their manifests:
$ diff -u /tmp/20180928.manifest /tmp/20181002.manifest | grep "^[-+]"
--- /tmp/20180928.manifest 2018-10-04 12:16:28.593773558 -0400
+++ /tmp/20181002.manifest 2018-10-04 12:16:57.945328943 -0400
-apparmor 2.12-4ubuntu7
+apparmor 2.12-4ubuntu8
-libapparmor1:amd64 2.12-4ubuntu7
+libapparmor1:amd64 2.12-4ubuntu8
-libnss-systemd:amd64 239-7ubuntu8
+libnss-systemd:amd64 239-7ubuntu9
-libpam-systemd:amd64 239-7ubuntu8
+libpam-systemd:amd64 239-7ubuntu9
-libsystemd0:amd64 239-7ubuntu8
+libsystemd0:amd64 239-7ubuntu9
-libudev1:amd64 239-7ubuntu8
+libudev1:amd64 239-7ubuntu9
-libzstd1:amd64 1.3.3+dfsg-2ubuntu1
+libzstd1:amd64 1.3.5+dfsg-1ubuntu1
-networkd-dispatcher 1.7-0ubuntu6
+networkd-dispatcher 1.7-0ubuntu8
-python3-configobj 5.0.6-2
+python3-configobj 5.0.6-3
-shared-mime-info 1.9-2
+shared-mime-info 1.10-1
-systemd 239-7ubuntu8
-systemd-sysv 239-7ubuntu8
+systemd 239-7ubuntu9
+systemd-sysv 239-7ubuntu9
-udev 239-7ubuntu8
+udev 239-7ubuntu9 |
|
| 2018-10-04 16:31:45 |
Scott Moser |
description |
The current squashfs/lxd image (20181002) is almost twice the size
(170M -> 323M) as the previous image (20180928) and boots dramatically
slower.
At first, just looking at manifests you'd blame systemd change, but I
think the real problem is seeded snaps.
$ lxc image list ubuntu-daily: | grep 18.10 | grep x86_64
| c (7 more) | 6cad1409aaac | yes | ubuntu 18.10 amd64 (daily) (20181002) | x86_64 | 323.85MB | Oct 2, 2018 at 12:00am (UTC) |
| | 21f94ea409ac | yes | ubuntu 18.10 amd64 (daily) (20180928) | x86_64 | 170.25MB | Sep 28, 2018 at 12:00am (UTC) |
| | 5011b07c621c | yes | ubuntu 18.10 amd64 (daily) (20180926) | x86_64 | 170.25MB | Sep 26, 2018 at 12:00am (UTC) |
## The old image
$ lxc launch ubuntu-daily:21f94ea409ac cosmic-test
$ lxc exec cosmic-test -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20180928
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/gyrNWkdSRG/
+ systemd-analyze
Startup finished in 11.822s (userspace)
graphical.target reached after 10.139s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @10.139s
└─multi-user.target @10.139s
└─snapd.seeded.service @6.041s +4.096s
└─snapd.service @5.770s +269ms
└─basic.target @5.698s
└─sockets.target @5.698s
└─snapd.socket @5.694s +3ms
└─sysinit.target @5.682s
└─cloud-init.service @4.676s +1.003s
└─systemd-networkd-wait-online.service @1.659s +3.015s
└─systemd-networkd.service @1.608s +49ms
└─network-pre.target @1.605s
└─cloud-init-local.service @198ms +1.406s
└─systemd-journald.socket @187ms
└─-.mount @184ms
└─system.slice @184ms
└─-.slice @184ms
## Try the current one.
$ lxc launch ubuntu-daily:cosmic c2
$ lxc exec c2 -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20181002
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/MCkrygb9sM/
+ systemd-analyze
Startup finished in 38.639s (userspace)
graphical.target reached after 36.990s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @36.990s
└─multi-user.target @36.989s
└─snapd.seeded.service @5.149s +31.839s
└─snapd.service @12.863s +67ms
└─snap-lxd-8959.mount @15.227s +15ms
└─snap.mount @11.543s +14ms
└─local-fs-pre.target @363ms
└─keyboard-setup.service @164ms +198ms
└─systemd-journald.socket @162ms
└─-.mount @159ms
└─system.slice @159ms
└─-.slice @159ms
Heres a diff of their manifests:
$ diff -u /tmp/20180928.manifest /tmp/20181002.manifest | grep "^[-+]"
--- /tmp/20180928.manifest 2018-10-04 12:16:28.593773558 -0400
+++ /tmp/20181002.manifest 2018-10-04 12:16:57.945328943 -0400
-apparmor 2.12-4ubuntu7
+apparmor 2.12-4ubuntu8
-libapparmor1:amd64 2.12-4ubuntu7
+libapparmor1:amd64 2.12-4ubuntu8
-libnss-systemd:amd64 239-7ubuntu8
+libnss-systemd:amd64 239-7ubuntu9
-libpam-systemd:amd64 239-7ubuntu8
+libpam-systemd:amd64 239-7ubuntu9
-libsystemd0:amd64 239-7ubuntu8
+libsystemd0:amd64 239-7ubuntu9
-libudev1:amd64 239-7ubuntu8
+libudev1:amd64 239-7ubuntu9
-libzstd1:amd64 1.3.3+dfsg-2ubuntu1
+libzstd1:amd64 1.3.5+dfsg-1ubuntu1
-networkd-dispatcher 1.7-0ubuntu6
+networkd-dispatcher 1.7-0ubuntu8
-python3-configobj 5.0.6-2
+python3-configobj 5.0.6-3
-shared-mime-info 1.9-2
+shared-mime-info 1.10-1
-systemd 239-7ubuntu8
-systemd-sysv 239-7ubuntu8
+systemd 239-7ubuntu9
+systemd-sysv 239-7ubuntu9
-udev 239-7ubuntu8
+udev 239-7ubuntu9 |
The current squashfs/lxd image (20181002) is almost twice the size:
170M -> 323M
as the previous image (20180928) and boots dramatically slower:
10s -> 38s
At first, just looking at manifests you'd blame systemd change, but I
think the real problem is seeded snaps.
$ lxc image list ubuntu-daily: | grep 18.10 | grep x86_64
| c (7 more) | 6cad1409aaac | yes | ubuntu 18.10 amd64 (daily) (20181002) | x86_64 | 323.85MB | Oct 2, 2018 at 12:00am (UTC) |
| | 21f94ea409ac | yes | ubuntu 18.10 amd64 (daily) (20180928) | x86_64 | 170.25MB | Sep 28, 2018 at 12:00am (UTC) |
| | 5011b07c621c | yes | ubuntu 18.10 amd64 (daily) (20180926) | x86_64 | 170.25MB | Sep 26, 2018 at 12:00am (UTC) |
## The old image
$ lxc launch ubuntu-daily:21f94ea409ac cosmic-test
$ lxc exec cosmic-test -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20180928
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/gyrNWkdSRG/
+ systemd-analyze
Startup finished in 11.822s (userspace)
graphical.target reached after 10.139s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @10.139s
└─multi-user.target @10.139s
└─snapd.seeded.service @6.041s +4.096s
└─snapd.service @5.770s +269ms
└─basic.target @5.698s
└─sockets.target @5.698s
└─snapd.socket @5.694s +3ms
└─sysinit.target @5.682s
└─cloud-init.service @4.676s +1.003s
└─systemd-networkd-wait-online.service @1.659s +3.015s
└─systemd-networkd.service @1.608s +49ms
└─network-pre.target @1.605s
└─cloud-init-local.service @198ms +1.406s
└─systemd-journald.socket @187ms
└─-.mount @184ms
└─system.slice @184ms
└─-.slice @184ms
## Try the current one.
$ lxc launch ubuntu-daily:cosmic c2
$ lxc exec c2 -- sh -xc 'cat /etc/cloud/build.info; dpkg-query --show | pastebinit; systemd-analyze; systemd-analyze critical-chain'
+ cat /etc/cloud/build.info
build_name: server
serial: 20181002
+ dpkg-query --show
+ pastebinit
http://paste.ubuntu.com/p/MCkrygb9sM/
+ systemd-analyze
Startup finished in 38.639s (userspace)
graphical.target reached after 36.990s in userspace
+ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @36.990s
└─multi-user.target @36.989s
└─snapd.seeded.service @5.149s +31.839s
└─snapd.service @12.863s +67ms
└─snap-lxd-8959.mount @15.227s +15ms
└─snap.mount @11.543s +14ms
└─local-fs-pre.target @363ms
└─keyboard-setup.service @164ms +198ms
└─systemd-journald.socket @162ms
└─-.mount @159ms
└─system.slice @159ms
└─-.slice @159ms
Heres a diff of their manifests:
$ diff -u /tmp/20180928.manifest /tmp/20181002.manifest | grep "^[-+]"
--- /tmp/20180928.manifest 2018-10-04 12:16:28.593773558 -0400
+++ /tmp/20181002.manifest 2018-10-04 12:16:57.945328943 -0400
-apparmor 2.12-4ubuntu7
+apparmor 2.12-4ubuntu8
-libapparmor1:amd64 2.12-4ubuntu7
+libapparmor1:amd64 2.12-4ubuntu8
-libnss-systemd:amd64 239-7ubuntu8
+libnss-systemd:amd64 239-7ubuntu9
-libpam-systemd:amd64 239-7ubuntu8
+libpam-systemd:amd64 239-7ubuntu9
-libsystemd0:amd64 239-7ubuntu8
+libsystemd0:amd64 239-7ubuntu9
-libudev1:amd64 239-7ubuntu8
+libudev1:amd64 239-7ubuntu9
-libzstd1:amd64 1.3.3+dfsg-2ubuntu1
+libzstd1:amd64 1.3.5+dfsg-1ubuntu1
-networkd-dispatcher 1.7-0ubuntu6
+networkd-dispatcher 1.7-0ubuntu8
-python3-configobj 5.0.6-2
+python3-configobj 5.0.6-3
-shared-mime-info 1.9-2
+shared-mime-info 1.10-1
-systemd 239-7ubuntu8
-systemd-sysv 239-7ubuntu8
+systemd 239-7ubuntu9
+systemd-sysv 239-7ubuntu9
-udev 239-7ubuntu8
+udev 239-7ubuntu9 |
|
| 2018-10-04 16:47:23 |
Scott Moser |
summary |
huge and slow image 20181002 |
huge and slow image 20181002 due to seeded lxd snap |
|
| 2018-10-04 16:55:25 |
Joshua Powers |
bug |
|
|
added subscriber Joshua Powers |
| 2018-10-04 20:37:46 |
Scott Moser |
cloud-images: status |
New |
Confirmed |
|
| 2018-10-04 20:37:49 |
Scott Moser |
cloud-images: importance |
Undecided |
High |
|
| 2018-10-04 21:40:11 |
Dan Watkins |
bug task added |
|
lxd (Ubuntu) |
|
| 2018-10-04 21:46:51 |
Robert C Jennings |
attachment added |
|
journalctl.txt https://bugs.launchpad.net/cloud-images/+bug/1796137/+attachment/5197328/+files/journalctl.txt |
|
| 2018-10-04 22:30:40 |
Patricia Gaughen |
bug |
|
|
added subscriber Dean Henrichsmeyer |
| 2018-10-09 23:27:09 |
William Grant |
bug |
|
|
added subscriber William Grant |
| 2018-10-15 16:48:49 |
Stéphane Graber |
bug |
|
|
added subscriber Ubuntu containers team |
| 2018-10-15 16:49:00 |
Stéphane Graber |
affects |
lxd (Ubuntu) |
snapd (Ubuntu) |
|
| 2019-10-09 20:30:15 |
Ian Johnson |
snapd (Ubuntu): status |
New |
Triaged |
|
| 2019-10-09 20:30:29 |
Ian Johnson |
snapd (Ubuntu): assignee |
|
Paweł Stołowski (stolowski) |
|
| 2019-10-09 20:30:39 |
Ian Johnson |
snapd (Ubuntu): importance |
Undecided |
High |
|
| 2023-03-15 19:45:07 |
John Chittum |
cloud-images: status |
Confirmed |
Fix Released |
|
