snaps do not start - cannot change mount namespace / cannot create writable mimic on openSUSE Tumbleweed
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
snapd |
Fix Committed
|
High
|
Unassigned | ||
apparmor (Suse) |
Fix Released
|
Medium
|
Bug Description
I updated my openSUSE Tumbleweed to version 20230613 (earlier it was a few days older), rebooted and now none of the snaps start.
All give errors like:
update.go:85: cannot change mount namespace according to change mount (/var/lib/
(tens of these)
and as a final one:
cannot update snap namespace: cannot create writable mimic over "/usr/lib/
snap-update-ns failed with code 1
---
Environment:
kernel 6.3.6-1-default
snapd rpm snapd-2.
snapd snap 2.59.4
Not sure of the causing change, at least snaps were working with older 6.3.x kernel. systemd just saw update 253.4 -> 253.5 which could be the one, or if 6.3.x introduced something in for example 6.3.5 or 6.3.6.
Recent changelogs:
20230613: https://<email address hidden>
20230612: https://<email address hidden>
20230610: https://<email address hidden>
20230608: https://<email address hidden>
20230607: https://<email address hidden>
20230605: https://<email address hidden>

|
#2 |

|
#3 |
Can confirm the same issue after latest dup (VERSION_
stratos@teras:~> winbox
cannot perform operation: mount -t tmpfs /tmp/snap.
stratos@teras:~> authy
cannot perform operation: mount -t tmpfs /tmp/snap.

|
#4 |
AppArmor 3.1.4 fixed a bug in mount rules - before they allowed things that the profile didn't really allow, and now they allow exactly what is specified in the profile. See https:/
This also means that profiles that "somehow worked" before now cause denials because they don't have the mount rules they really need.
https:/
type=AVC msg=audit(
Can you please confirm that you get a similar line in your audit.log when snap fails?
If I got the log message right, adding the following rule to the snap-confine profile should fix the problem:
mount fstype=tmpfs -> /tmp/snap.
That all said, I'll hand over the bug to the (system:

|
#5 |
(In reply to Christian Boltz from comment #2)
> https:/
> from /var/log/
>
> type=AVC msg=audit(
> class="mount" info="failed perms check" error=-13
> profile=
> pid=13661 comm="snap-confine" fstype="tmpfs" srcname="none"
>
> Can you please confirm that you get a similar line in your audit.log when
> snap fails?
Yes, have the same:
> telegram-desktop
cannot perform operation: mount -t tmpfs /tmp/snap.
> And in the log:
type=AVC msg=audit(
>
> If I got the log message right, adding the following rule to the
> snap-confine profile should fix the problem:
>
> mount fstype=tmpfs -> /tmp/snap.
Where to add this?

|
#6 |
(In reply to Christian Boltz from comment #2)
> Can you please confirm that you get a similar line in your audit.log when
> snap fails?
Can confirm the same message in audit.log:
type=AVC msg=audit(
type=BPF msg=audit(
> If I got the log message right, adding the following rule to the
> snap-confine profile should fix the problem:
>
> mount fstype=tmpfs -> /tmp/snap.
Yes it resolves the issue. I have added it in /etc/apparmor.
Thank you for the workaround.

|
#7 |
(In reply to Stratos Zolotas from comment #4)
> (In reply to Christian Boltz from comment #2)
> > If I got the log message right, adding the following rule to the
> > snap-confine profile should fix the problem:
> >
> > mount fstype=tmpfs -> /tmp/snap.
>
> Yes it resolves the issue. I have added it in
> /etc/apparmor.
Helped me as well - thank You all!

|
#8 |
This PR has been opened to fix the issue https:/

|
#9 |
I'll cherry pick this patch for the upcoming 2.59.5 update.

|
#10 |
Fun, the package status did not propagate to boo. Anyway, snapd 2.59.5 was pushed to the repository with the cherry pick included. If it's fixed for folks then fell free to close the bug.

|
#11 |
Can confirm the fix after latest update.

|
#12 |
I am facing again the same issue with all snaps (possibly not exactly the same problem but similar).
cat /etc/os-release
NAME="openSUSE Tumbleweed"
# VERSION="20230612"
ID="opensuse-
ID_LIKE="opensuse suse"
VERSION_
stratos@teras:~> winbox
update.go:85: cannot change mount namespace according to change mount (/run/user/
stratos@teras:~> authy
update.go:85: cannot change mount namespace according to change mount (/run/user/

|
#13 |
(In reply to Stratos Zolotas from comment #10)
> I am facing again the same issue with all snaps (possibly not exactly the
> same problem but similar).
Yes, I have exactly the same today!

|
#14 |
(In reply to Aleksey Kontsevich from comment #11)
> (In reply to Stratos Zolotas from comment #10)
> > I am facing again the same issue with all snaps (possibly not exactly the
> > same problem but similar).
>
> Yes, I have exactly the same today!
I don't think it's the same problem.
What I see is:
time->Wed Jun 14 13:18:59 2023
type=AVC msg=audit(
while there is exactly a rule allowing this in /var/lib/
mount options=(rbind, rw) "/snap/
AFAICT the profile is loaded:
maciek@sloop:~ sudo aa-status|grep ohmygira
snap-
snap.
snap.
Just to be extra sure I reloaded it again myself, and the effect is the same. I'm afraid someone with a deeper knowledge of apparmor is needed here.
description: | updated |

|
#15 |
Upstream AppArmor is aware of the new issue. If everything works out as planned, I'll get a patch and can offer a test package tomorrow.

|
#16 |
Hmmm... I have this error only for telegram, acestream works fine!

Timo Jyrinki (timo-jyrinki) wrote : | #1 |
There is discussion at: https:/
And SUSE bug at https:/
Changed in apparmor (Suse): | |
importance: | Unknown → Medium |
status: | Unknown → Confirmed |

|
#17 |
Unfortunately the issue turned out to be more complicated - and, worse, hard to reproduce. Therefore I'll forward a request from
https:/
what is the kernel version, and can you attach the full profile.
Therefore: Please attach the full profile of a snap that causes the mount denial, ideally both
- the profile file itsself, and
- the full profile with all includes inlined, which you can get with
/sbin/
/tmp/
(replace PROFILE_FILE with the actual filename)
Please also include your kernel version ("uname -a" output).

|
#18 |
> uname -a
Linux Aleksey 6.3.7-1-default #1 SMP PREEMPT_DYNAMIC Mon Jun 12 05:01:32 UTC 2023 (b5f9ff5) x86_64 x86_64 x86_64 GNU/Linux

|
#19 |
Created attachment 867626
snap.telegram-
This one causes problems

|
#20 |
Created attachment 867627
snap.acestreamp
this one works fine

|
#21 |
(In reply to Aleksey Kontsevich from comment #17)
> Created attachment 867626 [details]
> snap.telegram-
>
> This one causes problems
This profile does not contain any mount rules, and hence does not allow mount operations. The issue here is different than in #12, this one is snap not giving/generating the permission that is being denied.

|
#22 |
(In reply to Aleksey Kontsevich from comment #17)
> Created attachment 867626 [details]
> snap.telegram-
>
> This one causes problems
Can you provide the denial messages, this one is causing?

|
#23 |
(In reply to Aleksey Kontsevich from comment #18)
> Created attachment 867627 [details]
> snap.acestreamp
>
> this one works fine
Interestingly this one doesn't have any mount rules either. Which makes seeing the errors for the telegram profile in comment #17 even more important to try and figure out your issue.

|
#24 |
(In reply to John Johansen from comment #19)
> (In reply to Aleksey Kontsevich from comment #17)
> > Created attachment 867626 [details]
> > snap.telegram-
> >
> > This one causes problems
>
> This profile does not contain any mount rules, and hence does not allow
> mount operations. The issue here is different than in #12, this one is snap
> not giving/generating the permission that is being denied.
> ls /var/lib/
snap.acestreamp
snap.acestreamp
snap.acestreamp
snap-confine.
snap-confine.
snap.telegram-
snap.telegram-
snap-update-
snap-update-
Which one do You need?
(In reply to John Johansen from comment #20)
> (In reply to Aleksey Kontsevich from comment #17)
> > Created attachment 867626 [details]
> > snap.telegram-
> >
> > This one causes problems
>
> Can you provide the denial messages, this one is causing?
> telegram-desktop
update.go:85: cannot change mount namespace according to change mount (/var/lib/
update.go:85: cannot change mount namespace according to change mount (/var/lib/
update.go:85: cannot change mount namespace according to change mount (/snap/
update.go:85: cannot change mount namespace according to change mount (/snap/
update.go:85: cannot change mount namespace according to change mount (/snap/
update.go:85: cannot change mount namespace according to change mount (/snap/
update.go:85: cannot change mount namespace according to change mount (/snap/
update.go:85: cannot change mount namespace according to change mount (/snap/
update.go:85: cannot change mount namespace according to change mount (/snap/

|
#25 |
Created attachment 867630
snap-update-

|
#26 |
Created attachment 867631
snap-update-

|
#27 |
This issue is happening with Spotify and Brave as well.

|
#28 |
Created attachment 867632
/sbin/apparmor_
> uname -a
Linux localhost.
On above system with apparmor 3.1.5-1.2, trying to install chromium as below fails.
> snap install chromium --channel=
2023-06-
error: cannot perform the following tasks:
- Run configure hook of "chromium" snap if present (run hook "configure":
-----
update.go:85: cannot change mount namespace according to change mount (/var/lib/
update.go:85: cannot change mount namespace according to change mount (/var/lib/
update.go:85: cannot change mount namespace according to change mount (/var/snap/
cannot update snap namespace: cannot create writable mimic over "/snap/
snap-update-ns failed with code 1
-----)
On a related system with the same uname -a output, but with apparmoer 3.1.4-2.1 chromium installs and works fine. The attachment snap.chromium.

|
#29 |
Thanks Aleksey, can you also include kernel log info. I don't need everything just grep for DENIED
you should see some messages that look similar to
[ 1095.589947] audit: type=1400 audit(168688908

|
#30 |
Created attachment 867633
/sbin/apparmor_
> uname -a
Linux localhost.
> spotify
type=AVC msg=audit(
type=AVC msg=audit(
type=AVC msg=audit(

|
#31 |
Created attachment 867634
/sbin/apparmor_
> uname -a
Linux localhost.
> brave
type=AVC msg=audit(
type=AVC msg=audit(

|
#32 |
Created attachment 867635
/sbin/apparmor_
> uname -a
Linux localhost.
> authy
type=AVC msg=audit(
type=AVC msg=audit(
type=AVC msg=audit(
type=AVC msg=audit(
type=AVC msg=audit(
type=AVC msg=audit(
tform/" pid=26702 comm="5" srcname=
type=AVC msg=audit(
type=AVC msg=audit(

|
#33 |
(In reply to John Johansen from comment #27)
> Thanks Aleksey, can you also include kernel log info.
Command or path?
>I don't need everything just grep for DENIED

|
#34 |
(In reply to Aleksey Kontsevich from comment #31)
> (In reply to John Johansen from comment #27)
> > Thanks Aleksey, can you also include kernel log info.
>
> Command or path?
grep DENIED /var/log/
(if you don't have auditd running, grep DENIED /var/log/messages or the journalctl -b output)

|
#36 |
Created attachment 867652
kernel.log

|
#37 |
(In reply to Christian Boltz from comment #32)
> grep DENIED /var/log/
This one attached.

Ilgaz (ilgaz) wrote : | #35 |
- profile_with_includes Edit (125.2 KiB, text/plain)
Linux mbt 6.3.7-1-default #1 SMP PREEMPT_DYNAMIC Mon Jun 12 05:01:32 UTC 2023 (b5f9ff5) x86_64 x86_64 x86_64 GNU/Linux

|
#38 |
Same problem here with Authy and scrcpy.
openSuse Tumbleweed.
cris@polaris:~> uname -a
Linux polarisuse 6.3.7-1-default #1 SMP PREEMPT_DYNAMIC Mon Jun 12 05:01:32 UTC 2023 (b5f9ff5) x86_64 x86_64 x86_64 GNU/Linux
Cris

|
#39 |
Packages with the proposed upstream patch just finished building in my home repo. To get them, you can either add the repo
http://
or you can download the package x86_64/
(Only the apparmor-parser package changed, there's no need to install other packages from my home repo.)
After installling the (hopefully) fixed apparmor-parser package from home:cboltz, please test if snap now works, and report back.
(If you still notice failures, please attach the profile and the audit.log.)

|
#40 |
(In reply to Christian Boltz from comment #36)
> Packages with the proposed upstream patch just finished building in my home
> repo. To get them, you can either add the repo
> http://
> or you can download the package
> x86_64/
> in ./apparmor-
Shorter variant:
sudo zypper in http://
> After installling the (hopefully) fixed apparmor-parser package from
> home:cboltz, please test if snap now works, and report back.
> (If you still notice failures, please attach the profile and the audit.log.)
Not fixed, same errors for telegram.

|
#41 |
(In reply to Aleksey Kontsevich from comment #37)
> (In reply to Christian Boltz from comment #36)
> > After installling the (hopefully) fixed apparmor-parser package from
> > home:cboltz, please test if snap now works, and report back.
> > (If you still notice failures, please attach the profile and the audit.log.)
>
> Not fixed, same errors for telegram.
Sorry, telegram works now:
> sudo systemctl restart apparmor.service
did not helped for some reason, so forced to restart PC.

|
#42 |
(In reply to Aleksey Kontsevich from comment #38)
> > sudo systemctl restart apparmor.service
>
> did not helped for some reason, so forced to restart PC.
Wild guess: The snap profiles live in /var/lib/
"systemctl restart apparmor" will only reload the profiles in /etc/apparmor.d/, so you'll need to somehow force snap to reload the profile. Of course, rebooting is a way to do this, but maybe
apparmor_parser -r /var/lib/
is less annoying ;-)
(I don't use snap, therefore i don't know if there is a more "official" method to force a reload of its AppArmor profiles.)

|
#43 |
Thank you @ChristianBoltz, with your fixed package my snaps are working now.
As Aleksey told, `systemctl restart apparmor.service` did not help.
I also tried the command you suggested (apparmor_parser -r /var/lib/
After a reboot everything is working smoothly.

|
#44 |
> After installling the (hopefully) fixed apparmor-parser package from home:cboltz, please test if snap now works, and report back.
Brave, Spotify, Authy, VSCode, Opera and Slack are working with your package. Thanks for your effort to find a solution!

|
#45 |
(In reply to Christian Boltz from comment #36)
> Packages with the proposed upstream patch just finished building in my home
> repo. To get them, you can either add the repo
> http://
> or you can download the package
> x86_64/
> in ./apparmor-
>
> (Only the apparmor-parser package changed, there's no need to install other
> packages from my home repo.)
>
> After installling the (hopefully) fixed apparmor-parser package from
> home:cboltz, please test if snap now works, and report back.
> (If you still notice failures, please attach the profile and the audit.log.)
confirming that the apparmor-parser package from home:cboltz makes chromium from snap work again.

|
#46 |
(In reply to Cristiano Guadagnino from comment #40)
> Thank you @ChristianBoltz, with your fixed package my snaps are working now.
> As Aleksey told, `systemctl restart apparmor.service` did not help.
> I also tried the command you suggested (apparmor_parser -r
> /var/lib/
> After a reboot everything is working smoothly.
sorry, you would have needed to use
apparmor_parser -rT /var/lib/
the -T will cause apparmor to skip reading from the cache, forcing it to recompile the profiles. Since the neither the profile files nor the includes files were changed, apparmor will think the already compiled policy in the cache is still valid and load that.

|
#47 |
(In reply to Christian Boltz from comment #39)
> (In reply to Aleksey Kontsevich from comment #38)
> > > sudo systemctl restart apparmor.service
> >
> > did not helped for some reason, so forced to restart PC.
>
> Wild guess: The snap profiles live in /var/lib/
> right?
>
> "systemctl restart apparmor" will only reload the profiles in
> /etc/apparmor.d/, so you'll need to somehow force snap to reload the
> profile. Of course, rebooting is a way to do this, but maybe
> apparmor_parser -r /var/lib/
> is less annoying ;-)
> (I don't use snap, therefore i don't know if there is a more "official"
> method to force a reload of its AppArmor profiles.)
there isn't because an official way, ideally users shouldn't be tweaking/changing the snap generated profiles. You can try restarting the snapd.apparmor.
That could be worked around by manually deleting the profile cache, and then restarting the service.
With the newest versions of snapd vendoring apparmor, it might even be required to use the snapd.apparmor service because snap profiles might have policy rules that the system parser doesn't understand.

|
#48 |
Thanks for all the verifications. Upstream apparmor will roll a 3.1.6 release with the fix, so cboltz can get it released asap.

|
#49 |
AppArmor 3.1.6 has been released upstream. Thanks everyone for the reports and testing
https:/

|
#50 |
This is an autogenerated message for OBS integration:
This bug (1211989) was mentioned in
https:/
Changed in snapd: | |
status: | New → Fix Committed |
importance: | Undecided → High |

|
#51 |
The SR with AppArmor 3.1.6 was accepted and will be part of one of the next Tumbleweed snapshots.
For those who tested the apparmor-parser package from my home repo, please don't forget to switch back to the Tumbleweed package (as soon as 3.1.6 is available there).

|
#52 |
(In reply to John Johansen from comment #46)
> AppArmor 3.1.6 has been released upstream. Thanks everyone for the reports
> and testing
Works fine! Thanks to all!!!
Changed in apparmor (Suse): | |
status: | Confirmed → Fix Released |
Full description: /forums. opensuse. org/t/snap- applications- stopped- to-work- after-zypper- dup/166681 /forum. snapcraft. io/t/snap- applications- stopped- to-work- after-zypper- dup/35467 /forum. snapcraft. io/t/apparmor- issue/35461
- https:/
- https:/
- https:/