[SRU] Upgrade network-manager to latest point release

It's in a branch and needs sponsorship:
https://code.launchpad.net/~network-manager/network-manager/+git/ubuntu/+ref/happyaron/xenial-1.2.4

Aron, what do you think of including this in the SRU too?

manager-fix-state-transition-on-resuming-from-sleep.patch from Debian's 1.4.2-3

https://anonscm.debian.org/cgit/pkg-utopia/network-manager.git/commit/?id=3d4d73c6

If you want it, we'd need to add it to zesty and SRU it to yakkety also.

Looking at the NEWS, it looks like 1.2.6 is technically the latest 1.2 release, but it's up to you whether you want it in xenial now. The sleep patch I mentioned was included in 1.2.6.

@jbicha, would be nice if we can have 1.2.4 first, as backporting the patchsets to xenial could take a bit time sometimes (becasue we switched to git-pq in later release, thus noises of the patches).

I understand if you don't want 1.2.6, but the sleep patch should be easy to apply to xenial and yakkety and zesty. You can re-use this bug number and your sponsor can upload to all 3 series from here.

@jbicha, I would make updates to all 3 series quickly to introduce that fix, but it would be really nice to have an 1.2.4 update because it's been long waited (without postponing to wait for other series). What do you think?

Aron, please add the sleep patch to your master n-m branch for zesty and to your xenial branch. Make a yakkety branch or post a debdiff for yakkety.

I consider the sleep issue just as important as the 1.2.4 update. Therefore, I would rather not sponsor this update without that patch.

Then let's do the SRUs next week, my plan is to 1) have that patch in zesty 2) SRU 1.2.6 in yakkety 3) SRU 1.2.4 (with that patch on your request) in xenial. And after a short period of testing of 1.2.6 in yakkety push that point release to xenial. Does it sound good for you?

Sure, that's fine. Thanks!

@jbicha, here are the branches that need sponsor:

 * Zesty: master branch
    https://code.launchpad.net/~network-manager/network-manager/+git/ubuntu/+ref/master
 * Yakkety: yakkety SRU branch
    https://code.launchpad.net/~network-manager/network-manager/+git/ubuntu/+ref/yakkety
 * Xenial: happyaron/xenial-1.2.4 branch (would merge into xenial SRU branch after uploaded)
    https://code.launchpad.net/~network-manager/network-manager/+git/ubuntu/+ref/happyaron/xenial-1.2.4

Thank you Aron! I uploaded all 3 branches.

I pushed my changelog updates to https://git.launchpad.net/~jbicha/network-manager

I'm unsubscribing ubuntu-sponsors now. If there is anything else that needs sponsoring, feel free to resubscribe.

I guess this hasn't been acked from the queue because the sleep patch still hasn't landed zesty. There are some systemd test failures which don't seem to be caused by n-m though, so I'll ack the SRU's now so that they can be tested by real people..

Hello Aron, or anyone else affected,

Accepted network-manager into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/1.2.4-0ubuntu0.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Aron, or anyone else affected,

Accepted network-manager into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/1.2.6-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

After installing this upgrade I lost wifi networking after a sleep/resume cycle. Downgrading back to 1.2.2 fixed the issue. This is on a Dell XPS 9350 with an Intel Corporation Wireless 8260 (rev 3a).

Failed verification, will give another try next week.

Just to let know the other users who might search for the reason of networking regression in this place: after installation of this update yesterday, I observe on my laptop (Lenovo X220) a problem with wifi after a sleep/resume, as reported earlier by Dalila.

Symptoms:
- laptop won't connect automatically to wifi,
- network indicator shows only one (apparently) random SSID on the list of available networks,
- the problem is resolved with the command: sudo service network-manager restart

This problem existed for me since the upgrade to Xenial (16.04), but appeared only every 10 suspends or so. Now it happens every time, so I suppose it will be easier to find the reason and definitely solve the problem.

Mark as Fix Released in Zesty.

@ubuntu-sponsor, please help sponsor the following branch xenial, which has been updated to 1.2.6 as well:
https://code.launchpad.net/~network-manager/network-manager/+git/ubuntu/+ref/xenial

Thanks Aron, the update seems mostly fine but some small comments

- usually SRUs changes are kept to the minimum, I guess you can argue that the debian/rules change is safe and useful to you but unsure how much the SRU team is going to like that (it's probably fine though but we should at least try to describe the rational in the changelog)

- " * Patches merged upstream and dropped:
    - manager-fix-state-transition-on-resuming-from-sleep.patch"

Why do you comment it in the serie and keep it in source rather than cleaning it out?

- " * Patches restored for which is relevant again:
    - Don-t-block-network.target-on-NetworkManager-wait-on.patch"

Could you give some more rational for this change, in what situation is it useful? Ideally there would be a corresponding launchpad bug

The first points are nitpicks and shouldn't block the update but I think the restore patch need a bit more explanation, then looks fine to upload.

@Jeremy since you sponsored the previous version and were interested in some of the changes from Debian which are included do you maybe want to give the update a try to see if it address the issue you wanted to see fixed and then sponsor the changes?

- d/rules: yes that helped a lot for testing

- commenting out patch: in xenial branch we haven't switched to gbp-pq yet, and historically some patches were commented out when not used, and I'm following it

- it was believed not useful during a bunch update of n-m 1.2 packages (both debian and ubuntu), but later we switched back to hook up the service in WantedBy=network-online.target dynamically. This change makes it easier to disable the service and is what we have in yakkety, but the change can be dropped if it's not appropriate for SRU.

@jbicha, would be great to make another new upload to zesty, I've prepared an update of 1.4.4 in git repo:

https://code.launchpad.net/~network-manager/network-manager/+git/ubuntu/+ref/master

Given the large number of changes in a "backport" SRU and the changes to patches (looks like they are upstreamed, but maybe some bits are missing?), I think this bug is lacking in terms of a comprehensive set of test cases.

Some time ago I had captured the kind of tests I'd normally do when uploading NM: https://wiki.ubuntu.com/NetworkManager/DistroTesting. It's probably out of date, but it ought to be something you can base the test cases on.

Among other things, I'd carefully check that bridges work, and that they're not adversely affected when one uses virt-manager or lxd/lxc or something else on their system -- NM should definitely not mangle them, it should just ignore them. Similarly, VPNs should be checked carefully to make sure split tunnelling and "everything through the VPN" works correctly at least for OpenVPN (given DNS changes). We wouldn't want behavior worse than it was before.

@cyphermox, most of the common cases are covered before I push changes to the git repo. I've made some minor updates to NetworkManager/DistroTesting page and let's continue use it for smoke testing.

I've walked through the tests I can do and the yakkety version works, tagging verification-done-yakkety.

This bug was fixed in the package network-manager - 1.2.6-0ubuntu1

---------------
network-manager (1.2.6-0ubuntu1) yakkety; urgency=medium

  * Rebase to upstream point release 1.2.6 (LP: #1645698)
  * Refreshing patches, dropped ones that are merged upstream:
    - wifi-clear-WiFi-requested_scan-if-suppl-exits.patch
    - wifi-clear-WiFi-requested_scan-if-suppl-goes-INACTIV.patch

 -- Aron Xu <email address hidden> Tue, 20 Dec 2016 00:54:10 +0800

The verification of the Stable Release Update for network-manager has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Hello Aron, or anyone else affected,

Accepted network-manager into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/network-manager/1.2.6-0ubuntu0.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I tested 1.2.6-0ubuntu0.16.04.1 from xenial-proposed and it works for me with no problems and it also seems to fix #1585863

I also tested 1.2.6-0ubuntu0.16.04.1 and unfortunately it seems to break suspend on my system. Instead of suspending, the system goes into a forced reboot.

Harm, can you find any logs that indicate that network-manager has anything to do with that? Is that problem reliably reproducible with the updated network-manager but not with the older network-manager?

Hi Harm, could you please provide the detail information about the hardware which issue happened on?
ex.
* what's the platform brand and what's network devices?
* lspci -vvvnn, lsusb -vvv, and the /var/syslog

On closer examination the suspend issue that I was seeing was not related to the 1.2.6-0ubuntu0.16.04.1 networkmanager upgrade but a bios upgrade I performed a few days earlier. The forced reboot only occurred when suspending while powered and resuming on battery after unplugging the laptop, which is why I hadn't noticed earlier. After downgrading the bios and reinstalling the networkmanager update everything is working fine now.

Apologies for the false alarm. For anyone interested, the system is an XPS 9350 and the offending bios upgrade was 1.4.13 (reverting back to 1.4.12 fixed the issue).

I did test 1.2.6-0ubuntu0.16.04.1 and my observation is that the network applet is shown correctly and is functional as expected in about 70% of the resume situations. In the other cases killing and restarting the applet is a work around.

I also tested the network manager on Manjaro 16.10.3 and Korora 25. Both of them showed the behavior this SRU tries to fix.

My laptop:

Linux mark-Aspire-ES1-512 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64
x86_64 x86_64 GNU/Linux

Attached is the output of inxi -N, lsusb -vvv, lspci -vvvnn

inxi -N: Network: Card-1: Qualcomm Atheros QCA9565 / AR9565 Wireless Network Adapter
           driver: ath9k
           Card-2: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           driver: r8169

This bug was fixed in the package network-manager - 1.2.6-0ubuntu0.16.04.1

---------------
network-manager (1.2.6-0ubuntu0.16.04.1) xenial; urgency=medium

  * Rebase to upstream 1.2.6 release (LP: #1645698)
  * debian/rules: allow parallel building (copied from yakkety)
  * Dropped patch that was included in new release:
    - manager-fix-state-transition-on-resuming-from-sleep.patch
  * Restored Don-t-block-network.target-on-NetworkManager-wait-on.patch
    - This patch makes it easy to disable the service by tweaking the
      service file. The patch had been dropped because it didn't seem
      useful any more, but it was later re-introduced to both Debian and
      Ubuntu 16.10.
  * Refresh remaining patches

 -- Aron Xu <email address hidden> Thu, 12 Jan 2017 00:50:50 +0800

So, this managed to break at least my VPN setup quite well.

See #1672491 with the details, but the short version is that 'specify egress interface for each dnsmasq upstream server' breaks things in two cases.

The first could be argued to be a bug on the VPN client side, though a behavior change of this nature in an LTS release is decidedly unwelcome.

The second exposes a bug in dnsmasq which needs to be fixed for any case where an interface is removed and then readded, this includes suspend/resume and some VPN cases.

Regards,
Zephaniah E. Loss-Cutler-Hull.