systemd unit leaves a process running for the system lifetime but it only does anything at shutdown
Bug #1955084 reported by
Steve Langasek
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
unattended-upgrades (Ubuntu) |
New
|
High
|
Unassigned | ||
Jammy |
New
|
High
|
Unassigned |
Bug Description
The /lib/systemd/
tags: | added: rls-jj-incoming |
Changed in unattended-upgrades (Ubuntu Jammy): | |
importance: | Undecided → High |
tags: | added: fr-2007 |
tags: | removed: rls-jj-incoming |
To post a comment you must log in.
As requested, here's what I managed to dig up on possible alternatives upgrades- shutdown persistently:
to running unattended-
There appear to be several methods by which systemd can be told to run
something at or near shutdown, but one has to be exceptionally careful
in the selection to cover all edge cases.
Some possibilities can be trivially discarded. For instance, the
systemd-shutdown (8) logic runs too late, potentially after the root
file-system has been remounted read-only.
Other possibilities have extremely subtle side-effects. For example, one upgrades. service, which currently looks like this:
tempting possibility is to reverse the current logic of the
unattended-
[Service] /usr/share/ unattended- upgrades/ unattended- upgrade- shutdown --wait-for-signal c=1800
ExecStart=
KillMode=process
TimeoutStopSe
And instead use a "fake" service that systemd pretends is running, then
activate the shutdown script when systemd goes to stop the service, like
so:
[Service] /bin/true it=yes /usr/share/ unattended- upgrades/ unattended- upgrade- shutdown c=1800
ExecStart=
RemainAfterEx
ExecStop=
KillMode=process
TimeoutStopSe
Note the lack of --wait-for-signal (because we want the script to run
immediately when systemd goes to "stop" the fake service).
However, tempting though this possibility is, LP: #1778219 (and associated
commits, particularly [4]) describes why this doesn't work in practice. If a
package upgrade is occurring during shutdown, and that package attempts to
start or restart a systemd service, the attempt will hang. This is particularly
an issue with the "InstallOnShutdown" behaviour (not the default) which expects
to fully complete the upgrade before shutdown (as opposed to the default
behaviour which attempts to stop the upgrade after the next transaction
completes).
Side note: The KillMode setting appears to be an attempt to prevent systemd
from killing a subordinate apt process in the event it deems
unattended-upgrades to have timed out. This seems … risky to me. The timeout
duration is, so far as I can tell, picked fairly arbitrarily and what happens in the
event that unattended-upgrades times out, but leaves subordinate apt/dpkg
processes lying around is unclear (does systemd wait for them indefinitely,
does it kill them after another timeout, does it shutdown/reset the machine
with them still running?). There's some more context in LP: #1690980 regard
this.
Ultimately, the solution appears to have been having a persistent process
(u-u-shutdown) watch for the logind PrepareForShutdown event which occurs well
before systemd actually starts the shutdown. Unfortunately (as far as I've been
able to tell?) this requires an active process to watch for it.
The following bugs contain a great deal of detail and further context to this
issue:
* LP: #1778219 -- u-u hanging on shutdown
* LP: #1654600 -- u-u hanging when /var is separate file-system
* LP: #1618900 -- u-u failing to upgrade kernels due to missing mount (+others)
* LP: #1690980 -- context regarding u-u's shutdown inhibit, and killmode
There are also some useful test cases found in these bugs that should be used
in ensuring certain scenarios aren't broken. The f...