networkd-dhcp4 does not set prefsrc for dhcp-provided classless or static routes

upstream PR: https://github.com/systemd/systemd/pull/12970

test build in ppa:
https://launchpad.net/~ddstreet/+archive/ubuntu/lp1835581/+packages

Hello Dan, or anyone else affected,

Accepted systemd into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/240-6ubuntu5.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

I have accepted this to disco-proposed conditionally without the fix landing in eoan yet as per the ongoing 242 systemd merge, mostly because the proposed changes are already merged and accepted upstream. But to make sure eoan is not left without this fix, please be sure to push the changes to eoan-proposed as soon as the systemd merge is done, uploaded and migrated.

If possible, I'd prefer not releasing these out of -proposed without the eoan counterparts at least present in eoan-proposed. Thank you!

Hello Dan, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.25 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

autopkgtest failures:

systemd (disco/ppc64el):
  -storage test fails due to bug 1831459
  -upstream is currently flaky, bug 1831468, bug 1817721

linux/linux-kvm (bionic/various): these tests are totally unreliable

gvfs (bionic/ppc64el): tests are very flaky

network-manager (bionic/arm64): bug 1825946

Manual validate of fix in -proposed systemd on bionic, disco and eoan (eoan expected broken until followup to systemd-242 SRU is queued).

Azure uses DHCP classless routes and their backplane security blocks all secondary IP traffic to either their IMDS service or externally bound traffic.

Attached are success logs on Azure vms, showing proper handling of secondary static IPs and dhcp IP as preferred src on default routes for Bionic and Disco (and expected failure on Eoan) for this SRU.

Hello,

I've verified this with disco-updates.

/etc/systemd/network/80-ens3.network
::::::::::::::
[Match]
Name=ens3

[Network]
DHCP=ipv4
LinkLocalAddressing=ipv6
Address=172.16.0.5/24

root@disco-test:/home/ubuntu# ip -4 a show ens3
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 172.16.0.5/24 brd 172.16.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet 172.16.0.21/24 brd 172.16.0.255 scope global secondary dynamic ens3
       valid_lft 3485sec preferred_lft 3485sec

root@disco-test:/home/ubuntu# ip route
default via 172.16.0.1 dev ens3 proto dhcp src 172.16.0.21 metric 1024
172.16.0.0/24 dev ens3 proto kernel scope link src 172.16.0.5
172.16.0.1 dev ens3 proto dhcp scope link src 172.16.0.21 metric 1024

root@disco-test:/home/ubuntu# ip r get 1.1.1.1
1.1.1.1 via 172.16.0.1 dev ens3 src 172.16.0.21 uid 0
    cache

-> setting classless option

ubuntu@disco-test:~$ sudo su
root@disco-test:/home/ubuntu# ip -4 a show ens3
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 172.16.0.5/24 brd 172.16.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet 172.16.0.21/24 brd 172.16.0.255 scope global secondary dynamic ens3
       valid_lft 3575sec preferred_lft 3575sec
root@disco-test:/home/ubuntu# ip r
default via 172.16.0.1 dev ens3 proto dhcp metric 1024
172.16.0.0/24 dev ens3 proto kernel scope link src 172.16.0.5
root@disco-test:/home/ubuntu# ip r get 1.1.1.1
1.1.1.1 via 172.16.0.1 dev ens3 src 172.16.0.5 uid 0
    cache

-----> With the proposed package

ubuntu@disco-test:~$ sudo su
root@disco-test:/home/ubuntu# ip -4 a show ens3
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 172.16.0.5/24 brd 172.16.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet 172.16.0.21/24 brd 172.16.0.255 scope global secondary dynamic ens3
       valid_lft 3589sec preferred_lft 3589sec
root@disco-test:/home/ubuntu# ip r get 1.1.1.1
1.1.1.1 via 172.16.0.1 dev ens3 src 172.16.0.21 uid 0
    cache
root@disco-test:/home/ubuntu# ip r
default via 172.16.0.1 dev ens3 proto dhcp src 172.16.0.21 metric 1024
172.16.0.0/24 dev ens3 proto kernel scope link src 172.16.0.5

I've also verified that bionic-updates fixes the described issue

root@bionic-test:/home/ubuntu# ip route
default via 172.16.0.1 dev ens3 proto dhcp metric 1024
172.16.0.0/24 dev ens3 proto kernel scope link src 172.16.0.50
root@bionic-test:/home/ubuntu# ip r
default via 172.16.0.1 dev ens3 proto dhcp metric 1024
172.16.0.0/24 dev ens3 proto kernel scope link src 172.16.0.50
root@bionic-test:/home/ubuntu# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:db:d8:6a brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.50/24 brd 172.16.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet 172.16.0.24/24 brd 172.16.0.255 scope global secondary dynamic ens3
       valid_lft 3468sec preferred_lft 3468sec
    inet6 fe80::5054:ff:fedb:d86a/64 scope link
       valid_lft forever preferred_lft forever
root@bionic-test:/home/ubuntu# ip r
default via 172.16.0.1 dev ens3 proto dhcp metric 1024
172.16.0.0/24 dev ens3 proto kernel scope link src 172.16.0.50

root@bionic-test:/home/ubuntu# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:db:d8:6a brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.50/24 brd 172.16.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet 172.16.0.24/24 brd 172.16.0.255 scope global secondary dynamic ens3
       valid_lft 3435sec preferred_lft 3435sec
    inet6 fe80::5054:ff:fedb:d86a/64 scope link
       valid_lft forever preferred_lft forever
3: ens7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 52:54:00:11:c6:5b brd ff:ff:ff:ff:ff:ff
root@bionic-test:/home/ubuntu# ip r
default via 172.16.0.1 dev ens3 proto dhcp src 172.16.0.24 metric 1024
172.16.0.0/24 dev ens3 proto kernel scope link src 172.16.0.50
root@bionic-test:/home/ubuntu#

This is included in the current systemd in eoan-proposed,
https://launchpad.net/ubuntu/+source/systemd/243~rc1-0ubuntu1

This bug was fixed in the package systemd - 237-3ubuntu10.25

---------------
systemd (237-3ubuntu10.25) bionic; urgency=medium

  [ Dan Streetman ]
  * d/p/lp1835581-src-network-networkd-dhcp4.c-set-prefsrc-for-classle.patch:
    - set src address for dhcp 'classless' routes (LP: #1835581)
  * d/p/lp1833671-networkd-keep-bond-slave-up-if-already-attached.patch:
    - keep bond slave up if already attached (LP: #1833671)

  [ Jorge Niedbalski ]
  * d/p/lp1668771-resolved-switch-cache-option-to-a-tri-state-option-s.patch:
    Allows cache=no-negative option to be set, ignoring negative
    answers to be cached (LP: #1668771).

 -- Dan Streetman <email address hidden> Mon, 22 Jul 2019 12:45:02 -0400

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package systemd - 240-6ubuntu5.3

---------------
systemd (240-6ubuntu5.3) disco; urgency=medium

  [ Dan Streetman ]
  * d/p/lp1835581-src-network-networkd-dhcp4.c-set-prefsrc-for-classle.patch:
    - Set src address for dhcp 'classless' routes (LP: #1835581)

  [ Jorge Niedbalski ]
  * d/p/lp1668771-resolved-switch-cache-option-to-a-tri-state-option-s.patch:
    Allows cache=no-negative option to be set, ignoring negative
    answers to be cached (LP: #1668771).

 -- Dan Streetman <email address hidden> Mon, 22 Jul 2019 12:45:02 -0400

The systemd in eoan-proposed was version 243-rc1, which contained this, but that has been reverted and the current version is back to 240, which doesn't contain this. Discussion in #ubuntu-devel indicates eoan should eventually have at least version 241, so I'm going to wait for that, and then upload this fix if eoan doesn't already contain it.

This bug was fixed in the package systemd - 240-6ubuntu13

---------------
systemd (240-6ubuntu13) eoan; urgency=medium

  * Drop s390x seccomp fix causing regression on s390x
    Files:
    - debian/patches/src-shared-seccomp-util.c-Add-mmap-definitions-for-s390.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=da95e1d022e94a4d3ce0b69bd6eb398c95d09f24

 -- Balint Reczey <email address hidden> Mon, 26 Aug 2019 17:02:54 +0200

Hello Dan, or anyone else affected,

Accepted systemd into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/242-7ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted systemd (242-7ubuntu3.2) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

gvfs/1.42.1-1ubuntu1 (amd64)
systemd/242-7ubuntu3.2 (ppc64el)
ndctl/unknown (armhf)
casper/1.427 (amd64)
netplan.io/0.98-0ubuntu1 (ppc64el)
munin/unknown (armhf)
linux-oem-osp1/5.0.0-1026.29 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

dnsmasq server conf:

domain=test,1.2.3.4/24
dhcp-range=test,1.2.3.100,1.2.3.199,1m
dhcp-option=42,1.2.3.1
#dhcp-option=121,0.0.0.0/0,1.2.3.1

test system conf:

ubuntu@lp1835581-e:~$ cat /etc/systemd/network/10-ens8.network
[Match]
Name=ens8

[Network]
DHCP=ipv4
LinkLocalAddressing=ipv6
Address=1.2.3.99/24

ubuntu@lp1835581-e:~$ dpkg -l systemd|grep ii
ii systemd 242-7ubuntu3 amd64 system and service manager
ubuntu@lp1835581-e:~$ ip -4 a show ens8
3: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 1.2.3.99/24 brd 1.2.3.255 scope global ens8
       valid_lft forever preferred_lft forever
    inet 1.2.3.170/24 brd 1.2.3.255 scope global secondary dynamic ens8
       valid_lft 98sec preferred_lft 98sec
ubuntu@lp1835581-e:~$ ip -4 r
default via 1.2.3.1 dev ens8 proto dhcp src 1.2.3.170 metric 1024
1.2.3.0/24 dev ens8 proto kernel scope link src 1.2.3.99
1.2.3.1 dev ens8 proto dhcp scope link src 1.2.3.170 metric 1024

route src is correctly set to dhcp addr, when dnsmasq not configured with classless route

uncomment dnsmasq server conf classless route:

domain=test,1.2.3.4/24
dhcp-range=test,1.2.3.100,1.2.3.199,1m
dhcp-option=42,1.2.3.1
dhcp-option=121,0.0.0.0/0,1.2.3.1

reboot test system:

ubuntu@lp1835581-e:~$ ip -4 a show ens8
3: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 1.2.3.99/24 brd 1.2.3.255 scope global ens8
       valid_lft forever preferred_lft forever
    inet 1.2.3.170/24 brd 1.2.3.255 scope global secondary dynamic ens8
       valid_lft 110sec preferred_lft 110sec
ubuntu@lp1835581-e:~$ ip -4 r
default via 1.2.3.1 dev ens8 proto dhcp metric 1024
1.2.3.0/24 dev ens8 proto kernel scope link src 1.2.3.99

route src incorrectly not set, when dnsmasq set to classless route

change dnsmasq server again to comment out classless route and restart it, then reboot test system:

ubuntu@lp1835581-e:~$ dpkg -l systemd|grep ii
ii systemd 242-7ubuntu3.2 amd64 system and service manager
ubuntu@lp1835581-e:~$ ip -4 a show ens8
3: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 1.2.3.99/24 brd 1.2.3.255 scope global ens8
       valid_lft forever preferred_lft forever
    inet 1.2.3.170/24 brd 1.2.3.255 scope global secondary dynamic ens8
       valid_lft 97sec preferred_lft 97sec
ubuntu@lp1835581-e:~$ ip -4 r
default via 1.2.3.1 dev ens8 proto dhcp src 1.2.3.170 metric 1024
1.2.3.0/24 dev ens8 proto kernel scope link src 1.2.3.99
1.2.3.1 dev ens8 proto dhcp scope link src 1.2.3.170 metric 1024

re-enable classless route on server and reboot test system:

ubuntu@lp1835581-e:~$ ip -4 a show ens8
3: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 1.2.3.99/24 brd 1.2.3.255 scope global ens8
       valid_lft forever preferred_lft forever
    inet 1.2.3.170/24 brd 1.2.3.255 scope global secondary dynamic ens8
       valid_lft 104sec preferred_lft 104sec
ubuntu@lp1835581-e:~$ ip -4 r
default via 1.2.3.1 dev ens8 proto dhcp src 1.2.3.170 metric 1024
1.2.3.0/24 dev ens8 pro...

This bug was fixed in the package systemd - 242-7ubuntu3.2

---------------
systemd (242-7ubuntu3.2) eoan; urgency=medium

  [ Dan Streetman ]
  * d/extra/dhclient-enter-resolved-hook:
    - Replace use of bash-only &> with > and 2> (LP: #1849608)
  * d/p/lp1849658-resolved-set-stream-type-during-DnsStream-creation.patch:
    - Fix bug in refcounting TCP stream types (LP: #1849658)
  * d/extra/dhclient-enter-resolved-hook: cleanup temp $newstate file

  [ Rafael David Tinoco ]
  * Add support to KeepConfiguration= fixing behaviour for HA (LP: #1815101)
    - d/p/lp1815101-01-networkd-add-support-to-keep-configuration.patch
    - d/p/lp1815101-02-networkd-stop-clients-when-networkd-shuts-down.patch
    - d/p/lp1815101-03-network-add-KeepConfiguration-dhcp-on-stop.patch
    - d/p/lp1815101-04-network-make-KeepConfiguration-static-drop-DHCP-addr.patch
    - d/p/lp1815101-05-man-add-documentation-about-KeepConfiguration.patch

systemd (242-7ubuntu3.1) eoan; urgency=medium

  [ Balint Reczey ]
  * Fix shutdown and related actions from the login screen (LP: #1847896)
    File: debian/patches/logind-consider-greeter-sessions-suitable-as-display-sess.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=b407dfd8c9dc81594553c27467c35b383333d74c
  * debian/gbp.conf: Set debian-branch to ubuntu-eoan
    File: debian/gbp.conf
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=f399ce2cf4701a2dbb4b3505d2dd17a210d62f5c

  [ Dan Streetman ]
  * Fix bogus routes after DHCP lease change (LP: #1831787)
    Files:
    - debian/patches/lp1831787/0001-networkd-Add-back-static-routes-after-DHCPv4-lease-e.patch
    - debian/patches/lp1831787/0002-network-set-preferred-source-in-removing-route-entry.patch
    - debian/patches/lp1831787/0003-network-lower-log-level-about-critical-connection.patch
    - debian/patches/lp1831787/0004-network-reset-Link-dhcp4_configured-flag-earlier.patch
    - debian/patches/lp1831787/0005-network-split-dhcp_lease_lost-into-small-pieces.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ced3f5c2f619083f7beb164d94d4ccfe52222fe8
  * Set src address for dhcp 'classless' routes (LP: #1835581)
    File: debian/patches/lp1835581-src-network-networkd-dhcp4.c-set-prefsrc-for-classle.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=6a7ef370fb1335548448920be4ae6176b67044a8
  * Allows cache=no-negative option to be set, ignoring negative answers to
    be cached (LP: #1668771)
    File: debian/patches/lp1668771-resolved-switch-cache-option-to-a-tri-state-option-s.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=27def26f5b1d1b8ba314c4a925fc1b7c43837f86

 -- Dan Streetman <email address hidden> Fri, 01 Nov 2019 16:33:08 -0400