Upgrade fails when /var/lib/mysql is a soft link

The problem was an existing /var/lib/mysql-upgrade/DATADIR.link file, which caused the soft link backup mechanism to fail.

This can be fixed manually be deleting /var/lib/mysql-upgrade/DATADIR.link and running apt-get -f install.

To fix the preinst script, just change the line

cp -d "$checkdir" "$UPGRADEDIR/$dir.link"

to

cp -dT "$checkdir" "$UPGRADEDIR/$dir.link"

Fixed in upstream: https://github.com/ottok/mariadb-10.0/commit/93f4455d259a92efaa82a6557655395d565d25bb

Marking 'fix committed' as Ubuntu will soon sync latest version from Debian repos.

same bug when upgrading to 16.04 LTS with mysql

installation report:

% sudo apt-get install mysql-server
Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances
Lecture des informations d'état... Fait
The following additional packages will be installed:
  mysql-server-5.7
Paquets suggérés :
  mailx tinyca
Les NOUVEAUX paquets suivants seront installés :
  mysql-server mysql-server-5.7
0 mis à jour, 2 nouvellement installés, 0 à enlever et 0 non mis à jour.
Il est nécessaire de prendre 2 613 ko dans les archives.
Après cette opération, 48,6 Mo d'espace disque supplémentaires seront utilisés.
Souhaitez-vous continuer ? [O/n]
Réception de:1 http://fr.archive.ubuntu.com/ubuntu xenial/main amd64 mysql-server-5.7 amd64 5.7.11-0ubuntu6 [2 603 kB]
Réception de:2 http://fr.archive.ubuntu.com/ubuntu xenial/main amd64 mysql-server all 5.7.11-0ubuntu6 [10,1 kB]
2 613 ko réceptionnés en 1s (2 586 ko/s)
Préconfiguration des paquets...
(Lecture de la base de données... 251670 fichiers et répertoires déjà installés.)
Préparation du dépaquetage de .../mysql-server-5.7_5.7.11-0ubuntu6_amd64.deb ...
cp: impossible d'écraser le répertoire '/var/lib/mysql-upgrade/DATADIR.link/mysql' par un non répertoire
dpkg: erreur de traitement de l'archive /var/cache/apt/archives/mysql-server-5.7_5.7.11-0ubuntu6_amd64.deb (--unpack) :
 le sous-processus nouveau script pre-installation a retourné une erreur de sortie d'état 1
Sélection du paquet mysql-server précédemment désélectionné.
Préparation du dépaquetage de .../mysql-server_5.7.11-0ubuntu6_all.deb ...
Dépaquetage de mysql-server (5.7.11-0ubuntu6) ...
Des erreurs ont été rencontrées pendant l'exécution :
 /var/cache/apt/archives/mysql-server-5.7_5.7.11-0ubuntu6_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

% ls -ld /var/lib/mysql*
lrwxrwxrwx 1 mysql root 21 mars 16 17:16 /var/lib/mysql -> /home/Database/Mysql/
drwxr-xr-x 2 root root 4,0K avril 22 07:08 /var/lib/mysql-upgrade

Status changed to 'Confirmed' because the bug affects multiple users.

@racb would it be possible to get an SRU for this into 16.04 since the change is purely packaging-related, and it would be nice if the LTS was not broken in this way in perpetuity?

@pdf when we have a fix available, and depending on the invasiveness of the required fix, I would like to push it back to 16.04, yes. I'm focused on some other MySQL bugs that need fixing in 16.04 first though.

@racb, what is wrong with the fix itsadok described on 2015-07-20?

I've manually unpacked and applied that fix to workaround this issue a few times (every time there's an updated package).

I don't understand why it takes so long to get things like this fixed when there's solution posted at the top of the ticket.

Let's apply the "T" to the package's preinst and be done with it already. :P

@racb Is there any possibility that you apply the fix described at https://bugs.launchpad.net/ubuntu/+source/mariadb-10.0/+bug/1474212/comments/2 to mysql-server package?
It has been over a YEAR since this bug was open, the fix is known and works properly,
it just ONE letter in the script, it's not an upstream issue, it's just debian/ubuntu package bug.
I'm sick of fixing and manually repacking the deb every time an update arrives ...

Please get you lazy ass to fix this asap.

I had this on 160724 and 160623 on update of mysql-common

IMPACT:
hangs unattended apt-get upgrade
blocks package operations

GETAROUND:
rm /var/lib/mysql-upgrade/DATADIR.link
apt-get --fix-broken install

and SYMPTOM:
$ apt-get --yes dist-upgrade
…
Setting up mysql-common (5.7.13-0ubuntu0.16.04.2) ...
…
cp: cannot overwrite directory '/var/lib/mysql-upgrade/DATADIR.link/mysql' with non-directory
…
E: Sub-process /usr/bin/dpkg returned an error code (1)

and CAUSE:
mysqld --no-defaults --verbose --help | egrep ^datadir
datadir /var/lib/mysql/
ls -ld /var/lib/mysql
lrwxrwxrwx 1 mysql mysql 27 Mar 7 14:13 /var/lib/mysql -> /mnt/rwstore/rwstore/mysql/

If there are no plans to fix this, can we just stop releasing new updates of this package?

I personally don't need the unrelated mysql fixes and they just require us to re-patch mysql postinst again and again so the other dependent packages with actual fixes can satisfy their requirement for the latest mysql package revision.

Agree its frustrating that links don't work, especially as they hardcode directories in /usr/share/mysql/mysql-systemd-start (see bug/enhancement request #1611018)

Sorry, yes, this should have been fixed a long time ago

mysql-server-5.7_5.7.15-0ubuntu0.16.04.1 arrived with critical security fixes,
but this ONE LETTER bug is still not fixed.

How much do you want for fixing the bug that has a known fix FFS?
I'm willing to pay ...

We should have this fixed for the September milestone

Also, from the mysql-server-5.7_5.7.15-0ubuntu0.16.04.1 changelog:
  * SECURITY UPDATE: Update to 5.7.15 to fix security issues
    - CVE-2016-6662

:)

This bug was fixed in the package mysql-5.7 - 5.7.15-0ubuntu2

---------------
mysql-5.7 (5.7.15-0ubuntu2) yakkety; urgency=medium

  * Add working dir to perl lib path for dep8 upstream.
    New versions of perl will no longer automatically include working dir in
    the path. This was causing the mtr suite to fail to start.
  * Change mysql-testsuite dependency from python to libjson-perl.
    Tests written in python were rewritten in perl, so testsuite no longer
    depends on python, but tests fail if libjson-perl is missing.
  * Build with DWITH_LZ4=system
    If the flag is not set the package will be built with a bundled liblz4
    instead of the one provided by the liblz4-dev build-dep.
  * Add support for custom datadir to systemd service (LP: #1574782)
    The service was reporting errors if no database was found in /var/lib/mysql
    It will now check the path specified in the config
  * Fix support for soft-link datadir (LP: #1474212)
    Upgrading more than once when datadir is soft-link was causing errors
  * Escape special characters in password (LP: #1598992)
    The password string was not escaped, causing SQL errors if certain special
    characters such as apostrophes were included.
  * Removed always-false check on purge in postrm (LP: #1602945)
    Users were never seeing the query to remove data on a package purge

 -- Lars Tangvald <email address hidden> Fri, 16 Sep 2016 22:23:41 +0200

Otto, I'm not sure if this affects mariadb-10.0 on Xenial.

I still have this bug with mysql-server-5.7_5.7.15-0ubuntu0.16.04.1_amd64.deb downloaded today.

In my case /var/lib/mysql is a symlink that links to another partition.

xylo: Yes, we haven't ported it to Xenial yet. Note that there's a separate issue with having datadir set to the root of a partition (LP: #1580794), but that only affects fresh installs

Status changed to 'Confirmed' because the bug affects multiple users.

A bind mount rather than a link will resolve the issue.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

$ sudo apt-get update && sudo apt-get upgrade
Hit:1 http://se.archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://se.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:3 http://se.archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
You might want to run 'apt-get -f install' to correct these.
The following packages have unmet dependencies:
 mysql-server-5.7 : Depends: mysql-server-core-5.7 (= 5.7.16-0ubuntu0.16.04.1) but 5.7.17-0ubuntu0.16.04.1 is installed
E: Unmet dependencies. Try using -f.

$ sudo apt-get upgrade -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
  linux-image-extra-4.4.0-45-generic
Use 'sudo apt autoremove' to remove it.
The following packages will be upgraded:
  mysql-server-5.7
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
78 not fully installed or removed.
Need to get 0 B/2 856 kB of archives.
After this operation, 92,2 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Preconfiguring packages ...
(Reading database ... 166731 files and directories currently installed.)
Preparing to unpack .../mysql-server-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb ...
cp: cannot overwrite directory '/var/lib/mysql-upgrade/DATADIR.link/mysql' with non-directory
dpkg: error processing archive /var/cache/apt/archives/mysql-server-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb (--unpack):
 subprocess new pre-installation script returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/mysql-server-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Hello itsadok, or anyone else affected,

Accepted mysql-5.7 into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mysql-5.7/5.7.17-0ubuntu0.16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Verified fixed in 5.7.17-0ubuntu0.16.04.2

This bug was fixed in the package mysql-5.7 - 5.7.17-0ubuntu0.16.04.2

---------------
mysql-5.7 (5.7.17-0ubuntu0.16.04.2) xenial; urgency=medium

  * Add libjson-perl dependency for test suite (LP: #1631338)
    The last two python tests in the mtr suite were rewritten
    in perl, and require this package. Added for mysql-testsuite
    and as a build-dep.
  * Build with DWITH_LZ4=system (LP: #1631339)
    Package was previously built with bundled liblz4, though it
    was specified as a build-dep.
  * Add support for custom datadir to systemd service (LP: #1574782)
    The service was reporting an error if no database could be found in
    /var/lib/mysql. It now checks the location specified in the config.
    Note that user must still handle apparmor access for custom datadir.
  * Fix copy of soft-link datadir to /var/lib/mysql-upgrade (LP: #1474212)
    Upgrade would sometimes fail if mysql-upgrade already contained a
    link copy from a previous upgrade.
  * Escape special characters in password (LP: #1598992)
    Special characters in the root password would cause syntax errors
    and postinst failures.
  * Failing tests on platforms supported by upstream fail build (LP: #1646488)
    A passing test suite is now enforced on i386 and amd64 platforms at
    build-time.
  * d/copyright: Updated with information about new source files
  * Unstable test main.xa_prepared_binlog_off disabled pending upstream fix.
    Upstream bug report: http://bugs.mysql.com/bug.php?id=83340
  * d/lintian-overrides: Updated line numbers

 -- Lars Tangvald <email address hidden> Fri, 03 Feb 2017 08:53:04 +0100

The verification of the Stable Release Update for mysql-5.7 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.