[lunar] Snaps don't launch due to cgroup issue

https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1951491

Status changed to 'Confirmed' because the bug affects multiple users.

Interestingly, launching via snap run works, but not through GNOME.

In terminal with the direct app name too.
Seems random. Sometimes I can launch FF or any from dash.

Seems to be gone with core22 snap update.
Needs to be confirmed as solved by another user.

I still do experience some oddities (black window at start-up for TB, ...).

The bug is still present for me on core22 v20230304.

Yes for me too.

I'm also seeing this issue with Firefox & Thunderbird.

snap run also seems to consistently work

$ snap --version
snap 2.58.3+23.04ubuntu1
snapd 2.58.3+23.04ubuntu1
series 16
ubuntu 23.04
kernel 6.1.0-16-generic

Though if I click on Firefox then right click & open a new private window while it's "loading" (which it either will or it'll just spin and then not actually start) the private window seems to start fine. If I then click on Firefox it allows me to start a new non private window...

This is with both core22 20230304 (547) & core22 20230316 (583)

Very similar thing with Thunderbird...if I right click and 'compose new message' while it's "loading" I can then open a new window...

I can confirm the intermittent behavior described by Francois and Scott, as well as the ability to sometimes use New Private Window on the GNOME right click menu of FF during a loading period.

Sharing debug details from jamesh

When starting snap applications via the shell, log messages like the following are produced:
Mar 21 18:07:49 MGLT-Ubuntu chromium_chromium.desktop[13099]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-chromium_chromium-13099.scope is not a snap cgroup

This error is triggered in snapd’s device-cgroup-support.c, where it expects the process to be in a cgroup created by snapd but finds it in an app-gnome-*.scope cgroup instead. I think this is down to a race condition between gnome-shell and snapd.

On the gnome-shell side, it’s doing:

    spawns the app via GLib.spawn_async, which returns the pid of the running child process.

    Calls GnomeDesktop.start_systemd_scope on the pid.

    start_systemd_scope calls the systemd StartTransientUnit D-Bus method to move the pid to the app-gnome-*.scope cgroup.

On the snap side:

    the executables in /snap/bin are equivalent to running snap run snapname.

    the snap run code asks systemd to move its own pid to a snap.*.scope cgroup via StartTransientUnit.

    it then execs /usr/lib/snapd/snap-confine to set up the sandbox. This includes the device-cgroup-support.c code with the cgroup check.

    if it could set up the sandbox correctly, snap-confine execs the sandboxed application.

I think the problem occurs if gnome-shell’s StartTransientUnit call occurs between steps (2) and (3) on the snap side.

Status changed to 'Confirmed' because the bug affects multiple users.

Same problem here on Ubuntu 23.04 with wayland on AMD Ryzen 6850. Most snaps only start occasionally from dash icon. Works every time from console.

Not super how much it helps, but I have the same exact issue on two different systems (one using Kubuntu 22.10 -- with a mainline kernel of 6.0 and the other one using Kubuntu 23.04 [upgraded from 22.10] development branch with a mainline kernel of 6.2.7).

This specifically happens when krunner is used to launch an application, and it happens randomly (even in the same session; but I noticed that it is less likely to happen when the first session started and more likely to happen after you start a few apps).

Here is what I get when I run krunner and choose slack to launch.

❯ krunner --replace
/user.slice/user-1000.slice/user@1000.service/app.slice/app-slack_slack-7092918b60f045318ad5898c1cd14a71.scope is not a snap cgroup

First attempt fails with this, but then I try it again and it works (2nd attempt). I quite and try to launch it again, but this time first two attempts fail and the third one succeeds and launches slack again.

/user.slice/user-1000.slice/user@1000.service/app.slice/app-slack_slack-e699191c66c043c49b21af01db44a21f.scope is not a snap cgroup
/user.slice/user-1000.slice/user@1000.service/app.slice/app-slack_slack-a38502fd65ca425ebdf9b3cbac2d154c.scope is not a snap cgroup

❯ echo $DBUS_SESSION_BUS_ADDRESS
unix:path=/run/user/1000/bus

Everything I have is stock Kubuntu 23.04 (I'm updating it each day) except the mainline kernel (which is needed for the graphics driver).

System: AMD Ryzen 9 7950X as both the CPU and iGPU.

> I think this is down to a race condition between gnome-shell and snapd.

As someone who is using Kubuntu where gnome-shell is not present, I'd like to mention that I'm also experiencing the same exact issue. Not sure if there is a similar race condition is present on a KDE application in Kubuntu (like Konsole?) but this feels like more of a snap problem than a GNOME problem.

Adding snap version since I missed the chance of doing so in the last post:

❯ snap version
snap 2.58.3+23.04ubuntu1
snapd 2.58.3+23.04ubuntu1
series 16
ubuntu 23.04
kernel 6.2.7-060207-generic

Also I was able to record a trace of `SNAPD_DEBUG=1 krunner --replace slack` which includes a bit more information (not sure how useful)

Same issue here, I have never seen any snaps opening from the dash (both Firefox and Brave) and they both put out a cgroup error in the syslog. Starting from terminal works and without the cgroup error in the log.
Maybe also noticable, I see snap core20 and core22 both installed and cannot remove core20 because it is being used by firefox (but not brave which also does not start).

The list of snaps installed:

Name Version Rev Tracking Publisher Notes
bare 1.0 5 latest/stable canonical✓ base
brave 1.49.128 218 latest/stable brave -
core20 20230308 1852 latest/stable canonical✓ base
core22 20230316 583 latest/stable canonical✓ base
firefox 111.0-2 2453 latest/stable/… mozilla✓ -
gnome-42-2204 0+git.09673a5 65 latest/stable canonical✓ -
gtk-common-themes 0.1-81-g442e511 1535 latest/stable/… canonical✓ -
kde-frameworks-5-96-qt-5-15-5-core20 5.96.0 7 latest/stable kde✓ -
keepassxc 2.7.4 1645 latest/stable keepassxreboot -
libreoffice 7.5.1.2 270 latest/stable canonical✓ -
snap-store 41.3-76-g2e8f3b0 935 latest/stable/… canonical✓ -
snapd 2.58.3 18596 latest/stable canonical✓ snapd
snapd-desktop-integration 0.1 71 latest/stable/… canonical✓ -

We're also tracking a crash in bug 2012839 (private for now) which might explain similar issues seen outside of gnome-shell.

@Batuhan: it's different code, but looks to be the same type of problem:

https://github.com/KDE/kio/blob/5bda428992cbd168fc1898661d80a089a8217449/src/gui/systemd/scopedprocessrunner.cpp#L13-L41

It's moving the process to a new cgroup after the application has started executing (so happens without any synchronisation with what the application is doing).

Another idea for how to synchronise the StartTransientUnit call without resorting to passing pipe file descriptors to the child: we could simply have the child_setup_func call kill(getpid(), SIGSTOP). The parent process can now do the cgroup move at its leisure, and then send a SIGCONT to the child to let it continue to the exec() call.

The workaround that is coming in gnome-shell is enough to handle the normal snap cases, however that's not to be considered a proper solution.

The usage of `ChildSetupFunc` is not easily doable without some refactor in the shell that I'm not keen to do for something that is not likely going to be upstreamable, because the child PID isn't known at JS level at that point so we can't just use it without implementing something else in C ("it is not safe to call any function which may call malloc()" from there, so we can't control what gjs is going to do).

Even doing it, this wouldn't fix the majority of other cases where the cgroup move happens when using the launch context (i.e. when apps are launched from dock or activities).

So, I feel snapd tracker should somewhat be able to track that an app changed its cgroup and move it back to what it belongs, but as discussed in https://github.com/systemd/systemd/issues/26925 there's nothing telling us that right now.

This bug was fixed in the package gnome-shell - 44.0-2ubuntu2

---------------
gnome-shell (44.0-2ubuntu2) lunar; urgency=medium

  * debian/patches: Cherry-pick more upstream fixes from stable branch
  * debian/patches: Give more priority to desktop files matching startup-wm
    class (LP: #2012645)
  * debian/patches: Do not move snap apps to GNOME apps scope (LP: #2011806)

 -- Marco Trevisan (Treviño) <email address hidden> Mon, 27 Mar 2023 21:19:20 +0200

Fix works for me (after reboot/reload Gnome), fast job, thanks! :)

I still have an issue. Not sure if it is related.

When clicking the Firefox icon in the dock, a completely black window opens. Then I can close it and try again, and the second time it works as expected. This repeats itself in each session.

Same here.

I got it too but definitely another issue.

Il mar 28 mar 2023, 19:40 Gunnar Hjalmarsson <email address hidden>
ha scritto:

> I still have an issue. Not sure if it is related.
>
> When clicking the Firefox icon in the dock, a completely black window
> opens. Then I can close it and try again, and the second time it works
> as expected. This repeats itself in each session.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/2011806
>
> Title:
> [lunar] Snaps don't launch due to cgroup issue
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/2011806/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: distribution=ubuntu; sourcepackage=gnome-shell;
> component=main; milestone=ubuntu-23.04-beta; status=Fix Released;
> importance=Critical; assignee=mail@3v1n0.net;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=kio; component=universe;
> status=Confirmed; importance=Critical; assignee=None;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=snapd; component=main;
> status=Confirmed; importance=Critical; assignee=None;
> Launchpad-Bug-Tags: dt-1154 lunar rls-ll-incoming
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: 3v1n0 cmmrandau cooper48k fthx gunnarhj jamesh
> janitor kde-batuhan mikethebos1 requist1 seb128 vanvugt
> Launchpad-Bug-Reporter: Francois Thirioux (fthx)
> Launchpad-Bug-Modifier: Gunnar Hjalmarsson (gunnarhj)
> Launchpad-Message-Rationale: Assignee
> Launchpad-Message-For: 3v1n0
>
>

The black window is bug 1987976.

> So, I feel snapd tracker should somewhat be able to track that an app
> changed its cgroup and move it back to what it belongs, but as discussed
> in https://github.com/systemd/systemd/issues/26925 there's nothing
> telling us that right now.

There is no "snapd tracker". It's the "snap run" code that moves itself to a new cgroup before setting up the sandbox, dropping privileges, and executing the confined application. At that point, it has no permission to talk to systemd.

It's basically working like "systemd-run --scope" would: it sets up the environment appropriately and then lets the app take over.

I've filed https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/6565 upstream to cover the gnome-shell problem.

> There is no "snapd tracker".

Yeah, I mean not part of the daemon itself, I was referring to this piece of code:
 - https://github.com/snapcore/snapd/blob/master/sandbox/cgroup/tracking.go

But I see why it can't adjust things itself.
Thanks for the report upstream.

I did a search through the KDE bug tracker, and it looks like someone reported the same root problem back in 2020:

https://bugs.kde.org/show_bug.cgi?id=423756

Can confirm it affects KDE users as well - discovered this while trying to launch Discord for the first time in a long time. The snap will only launch from a terminal, not the shell ("not a snap cgroup error"). Flatpak version launches from shell just fine.

Any updates with this one?

When I launch Thunderbird

Nov 14 20:12:55 systemd[9205]: Started snap.thunderbird.thunderbird-218c47bb-9cdf-4585-8487-1e76547933b3.scope.
Nov 14 20:12:55 systemd[9205]: Started app-thunderbird_thunderbird-4758a597c99d427f91814da5b05c365a.scope - Thunderbird Mail - Mail Client.
Nov 14 20:12:55 plasmashell[133984]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-thunderbird_thunderbird-4758a597c99d427f91814da5b05c365a.scope is not a snap cgroup

Operating System: Kubuntu 23.10
KDE Plasma Version: 5.27.9
KDE Frameworks Version: 5.112.0
Qt Version: 5.15.10
Kernel Version: 6.5.0-10-generic (64-bit)
Graphics Platform: X11
Processors: 24 × AMD Ryzen 9 5900X 12-Core Processor
Memory: 62.7 GiB of RAM
Graphics Processor: AMD Radeon RX 6900 XT
Manufacturer: Gigabyte Technology Co., Ltd.
Product Name: X570S AORUS MASTER
System Version: -CF

This is also happening for me. It takes multiple tries to launch some of my snap applications like Discord, Zoom, and Slack from my Application Menu. I don't have this problem for IntelliJ Ultimate. It seemed like a lot of the applications that are having this issue rely on core22, but I don't know if that's actually related.

Here are my snaps (I've been trying to add and remove slack and zoom to try and fix them):

Name Version Rev Tracking Publisher Notes
bare 1.0 5 latest/stable canonical✓ base
core 16-2.60.4 16202 latest/stable canonical✓ core
core18 20230901 2796 latest/stable canonical✓ base
core22 20230801 864 latest/stable canonical✓ base
discord 0.0.35 163 latest/stable snapcrafters✪ -
gnome-42-2204 0+git.ff35a85 141 latest/stable canonical✓ -
gtk-common-themes 0.1-81-g442e511 1535 latest/stable canonical✓ -
gtk2-common-themes 0.1 13 latest/stable canonical✓ -
intellij-idea-ultimate 2023.2.5 462 latest/stable jetbrains✓ classic
snapd 2.60.4 20290 latest/stable canonical✓ snapd

If I try opening them directly from the terminal it works every time.

But if I open them using dolphin (with terminal open) I sometimes get the following error:

/user.slice/user-1000.slice/user@1000.service/app.slice/app-discord_discord-f458540dedf040da87633a4e587169ac.scope is not a snap cgroup

This seems to be consistent with the frequency of if I try and launch if from the Application Menu, so I'm assuming this is a similar error.

Here is my system information:

Operating System: Ubuntu 22.04
KDE Plasma Version: 5.27.9
KDE Frameworks Version: 5.104.0
Qt Version: 5.15.3
Kernel Version: 6.2.16-060216-generic (64-bit)
Graphics Platform: X11
Processors: 32 × AMD Ryzen 9 7950X 16-Core Processor
Memory: 30.5 GiB of RAM
Graphics Processor: Radeon RX 7900 XTX
Manufacturer: ASUS

Hopefully this information is useful in trying to find a fix!

@Jonas: If KDE is launching apps in the same way gnome-shell was, it is inherently racy. It needs to make sure the child process is in the new cgroup before executing the application.

Doing the cgroup move after the app is already running causes problems if the application itself wants to configure cgroups, and could let child processes of the app itself escape the cgroup if they are created before the launcher tries to move the app.

so...what's the fix?

My Bitwarden is also acting up now

So, this won't be fixed until https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/6565 is resolved? Or is it fixed in newer Ubuntu releases? (On 22.04 I am experiencing the problem quite often, wondering if there is any point in upgrading. In the last 24h I encountered it three times with Thunderbird, each time I could not start it for about ~20 minutes)

mar 05 22:11:42 pc thunderbird_thunderbird.desktop[1018965]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1018965.scope is not a snap cgroup
mar 05 22:12:13 pc thunderbird_thunderbird.desktop[1018995]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1018995.scope is not a snap cgroup
mar 05 22:12:32 pc thunderbird_thunderbird.desktop[1019063]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1019063.scope is not a snap cgroup
mar 05 22:13:36 pc thunderbird_thunderbird.desktop[1019106]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1019106.scope is not a snap cgroup
mar 05 22:15:39 pc thunderbird_thunderbird.desktop[1019468]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1019468.scope is not a snap cgroup
mar 05 22:17:04 pc thunderbird_thunderbird.desktop[1019559]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1019559.scope is not a snap cgroup
mar 05 22:19:04 pc thunderbird_thunderbird.desktop[1019660]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1019660.scope is not a snap cgroup
mar 05 23:24:55 pc thunderbird_thunderbird.desktop[1027932]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1027932.scope is not a snap cgroup
mar 05 23:29:02 pc thunderbird_thunderbird.desktop[1029016]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1029016.scope is not a snap cgroup
mar 06 00:00:09 pc thunderbird_thunderbird.desktop[1041966]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1041966.scope is not a snap cgroup
mar 06 00:00:33 pc thunderbird_thunderbird.desktop[1042002]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1042002.scope is not a snap cgroup
mar 06 00:01:06 pc thunderbird_thunderbird.desktop[1042035]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1042035.scope is not a snap cgroup
mar 06 00:01:22 pc thunderbird_thunderbird.desktop[1042063]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1042063.scope is not a snap cgroup
mar 06 00:01:52 pc thunderbird_thunderbird.desktop[1042091]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1042091.scope is not a snap cgroup
mar 06 00:03:16 pc thunderbird_thunderbird.desktop[1042381]: /user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-thunderbird_thunderbird-1042381.scope is not a snap cgroup
mar 06 00:04:17 pc thunderbird_thunderbird.desktop[1...

I am using latest Ubuntu version today (23.10 mantic) and it is not working. How is possible that something so basic as Ubuntu remote desktop with vnc just does not work or take so long. I am feed up of asking google. I remember years ago I just desist of doing a remote connection using vnc, cause I could not due Wayland was a new implantation. Today i arrived further away, but it still not working properly.