various linux packages being marked as manually installed, still prevents 'apt-get autoremove' from doing the right thing for kernels

Michael, I know we've discussed before about how to possibly fix this, but it seems this has yet to be implemented. Can you please look at whether this can be implemented in time for 15.04? This continues to be a source of pain for our users, who receive 40MB of new kernel in /boot every 3 weeks.

Status changed to 'Confirmed' because the bug affects multiple users.

Yes I am definitely seeing this behavior on Vivid with the latest kernel updates. Doing a regular apt-get dist-upgrade works fine and marks the new kernel packages as automatically installed. Using the update-manager always marks them as manually installed.

The reasons to why old kernels are not intentionally being marked for autoremoval (as of 2008-11-04) is because it was considered too dangerous, and apt has an exception to not tag linux-image-* packages for autoremoval because _we_ had no idea which kernels actually worked for a system.

However, I had thought that this problem had been addressed using last-good-boot for grub after runlevel 2 was reached, here:

https://wiki.ubuntu.com/KernelTeam/removing-old-kernels

But since the wiki hasn't been updated since '08, maybe that is not true anymore.

On Tue, Jul 07, 2015 at 02:37:07PM -0000, Tyler Dinsmoor wrote:
> The reasons to why old kernels are not intentionally being marked for
> autoremoval (as of 2008-11-04) is because it was considered too
> dangerous,

No, absolutely not. The reason they're not being marked for autoremoval is
because there's a bug in the interface between update-manager and aptdaemon.

I just ran dist-upgrade on my Kubuntu 14.04 system, and when it finished I had four kernels and associated files in /boot:
3.16.0.33-generic
3.16.0.38-generic
3.16.0.43-generic
3.16.0.44-generic

Autoremove failed to identify any of these. I had to run apt-get remove for the 33 and 38 kernels.

The attached patches seems to work, I just tested the upgrade and get:
# apt list linux-image-extra-3.19.0-28-generic
Listing... Done
linux-image-extra-3.19.0-28-generic/vivid-proposed,now 3.19.0-28.30 amd64 [installed,automatic]

I guess one could argue about a better API for the aptdaemon client for this, maybe "MarkAutoInstalled(pkg_list)" or Mark({Auto,Hold}, pkg_list".

I just raised bug #1492709 about a similar but not obviously identical issue on Trusty (auto-marking only works if updates are set to automatically install). Is it the same issue?

Opening a bug task on the aptdaemon upstream project for this as well, since solving this involves making changes to the aptdaemon protocol. Sebastian, or other aptdaemon maintainers, do you have any thoughts here on mvo's proposed implementation of taking this optional 'auto' flag from clients for packages being installed?

Since this is a longstanding critical bug, in the absence of any response I'm going to go ahead with uploading both the aptdaemon and update-manager changes to wily, with an eye to SRUing it to trusty and vivid later.

This does break compatibility with older aptdaemon, so I am adding a versioned Depends from update-manager. If the aptdaemon patch is rejected upstream and needs to be reverted in Ubuntu (in favor of a different solution), then some versioned Breaks will be needed at that time.

This bug was fixed in the package aptdaemon - 1.1.1+bzr982-0ubuntu13

---------------
aptdaemon (1.1.1+bzr982-0ubuntu13) wily; urgency=medium

  * Drop Vcs-Bzr from debian/control:
    - The branch pointed to is "ubuntu-vivid". This is obviously incorrect
      for wily; and you shouldn't have to do a busywork update of Vcs-Bzr
      with each release.
    - Uploaders do not have commit rights on the listed branch. A Vcs-Bzr
      field should only point to a branch that uploaders have rights to,
      otherwise it can never be authoritative.
  * debian/patches/lp1439769-aptdaemon-autoinstall.patch: Take a flag to
    indicate whether a requested package is auto-installed. Thanks to
    Michael Vogt <email address hidden>. Closes LP: #1439769.

 -- Steve Langasek <email address hidden> Tue, 06 Oct 2015 11:33:41 -0700

This bug was fixed in the package update-manager - 1:15.10.3

---------------
update-manager (1:15.10.3) wily; urgency=medium

  * Pass a '#auto' suffix to aptdaemon for packages that are autoinstalled,
    so that we have enough information to autoremove them later. Thanks to
    Michael Vogt <email address hidden> for the patch. LP: #1439769.
  * Add versioned dependency on aptdaemon (>= 1.1.1+bzr982-0ubuntu13) for the
    above.

 -- Steve Langasek <email address hidden> Tue, 06 Oct 2015 21:26:35 -0700

Steve Langasek, there is still no fix released for Trusty (and maybe Precise?). Bug #1492709

See also Bug #1175637

SRU for 14.04 maybe?

...so that unattended-upgrades / 'sudo apt-get autoremove' could remove extra kernels installed by Software Updater.

... that would be a great advantage, if /boot is about full, allowing installation of further kernel updates.

Status changed to 'Confirmed' because the bug affects multiple users.

This is fixed in Xenial and later in LP: #1675079