error loading realtime clock since upgrading to 17.10

Bug #1743027 reported by Peter Milley
32
This bug affects 6 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Confirmed
Medium
Unassigned

Bug Description

(originally asked over at askubuntu.com, but is still unresolved: https://askubuntu.com/questions/971349/problem-with-real-time-clock-since-upgrading-to-ubuntu-17-10)

I recently upgraded from Ubuntu 17.04 to Ubuntu 17.10 using the software update process. The update completed without any errors, and Ubuntu 17.10 is now running without any major issues.

However, I was running a script previously that uses rtcwake which has since stopped working. The script automatically hibernates my laptop after a set period of time in sleep mode, and was working perfectly in Ubuntu 17.04.

In the process of troubleshooting, I discovered that the problem was with rtcwake which is complaining about /dev/rtc0 missing.

Here is output from a sample rtcwake command:

peter@haven:~$ rtcwake -m no -s 1300
rtcwake: assuming RTC uses UTC ...
rtcwake: /dev/rtc0: unable to find device: No such file or directory

I verified that in fact, that directory is missing. Since this was working prior to the upgrade, I have to assume that it was there previously, but I don't know for sure.

In researching the error, I found some suggestions to get further information with timedatectl and hwclock, so here are the results from those commands as well.

Output of timedatectl:

peter@haven:~$ timedatectl
      Local time: Fri 2017-10-27 19:06:26 EDT
  Universal time: Fri 2017-10-27 23:06:26 UTC
        RTC time: n/a
       Time zone: America/New_York (EDT, -0400)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

Output of hwclock --debug:

peter@haven:~$ sudo hwclock --debug
[sudo] password for peter:
hwclock from util-linux 2.30.1
Trying to open: /dev/rtc0
Trying to open: /dev/rtc
Trying to open: /dev/misc/rtc
No usable clock interface found.
hwclock: Cannot access the Hardware Clock via any known method.

UPDATE

When I boot with my old kernel, 4.10.0-37-generic the real-time clock works fine.

I get the following RTC messages in dmesg when I boot the 4.13.0-16-generic kernel:

peter@haven:~$ dmesg | grep -i rtc
[ 0.089393] RTC time: 17:03:55, date: 11/01/17
[ 1.238784] rtc_cmos 00:01: RTC can wake from S4
[ 1.238794] rtc_cmos: probe of 00:01 failed with error -16
[ 1.295459] hctosys: unable to open rtc device (rtc0)

UPDATE #2

Well, I was starting to believe that I had a hardware issue, since discovering I was unable to save changes to my BIOS and could not boot from USB. As it turns out, my laptop was bitten by this bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1734147 causing my BIOS to be locked in a read-only state.

Once I resolved the BIOS issue, and fixed the USB boot problem, I did a live boot of Ubuntu 17.10. The issue with my real time clock still exists under this clean boot scenario, yet when I boot other distros, like Linux Mint 18.3 and the real time clock works fine. I feel this definitely has something to do with the Ubuntu install, perhaps even a kernel bug.

UPDATE #3

More evidence to suggest this is a kernel bug: I live-booted a copy of Antergos Linux, which was using the Arch kernel 4.14.12-1-ARCH and it exhibited the same problem with the real time clock as Ubuntu 17.10. In addition, I confirmed that the Linux Mint version I booted earlier (see Update #2) was on the Ubuntu kernel 4.10.0-38-generic.

So, I suspect that sometime between kernels 4.10 and 4.13 this bug was introduced.
---
ApportVersion: 2.20.7-0ubuntu3.7
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: peter 1832 F.... pulseaudio
 /dev/snd/controlC1: peter 1832 F.... pulseaudio
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 17.10
HibernationDevice: RESUME=UUID=a602efe1-467d-4ac4-806f-1b931c91cdad
InstallationDate: Installed on 2017-04-05 (292 days ago)
InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64 (20161012.2)
MachineType: LENOVO 20354
Package: linux (not installed)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.13.0-25-generic.efi.signed root=UUID=fa5454aa-9072-475d-b515-acd51a6166cc ro quiet splash vt.handoff=7
ProcVersionSignature: Ubuntu 4.13.0-25.29-generic 4.13.13
RelatedPackageVersions:
 linux-restricted-modules-4.13.0-25-generic N/A
 linux-backports-modules-4.13.0-25-generic N/A
 linux-firmware 1.169.1
Tags: wayland-session artful
Uname: Linux 4.13.0-25-generic x86_64
UpgradeStatus: Upgraded to artful on 2017-10-24 (90 days ago)
UserGroups: adm cdrom dip kismet lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 07/21/2015
dmi.bios.vendor: LENOVO
dmi.bios.version: 9BCN91WW
dmi.board.asset.tag: No Asset Tag
dmi.board.name: Lancer 5A5
dmi.board.vendor: LENOVO
dmi.board.version: 31900058WIN
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Lenovo Z50-70
dmi.modalias: dmi:bvnLENOVO:bvr9BCN91WW:bd07/21/2015:svnLENOVO:pn20354:pvrLenovoZ50-70:rvnLENOVO:rnLancer5A5:rvr31900058WIN:cvnLENOVO:ct10:cvrLenovoZ50-70:
dmi.product.family: IDEAPAD
dmi.product.name: 20354
dmi.product.version: Lenovo Z50-70
dmi.sys.vendor: LENOVO

Revision history for this message
Peter Milley (pbmilley) wrote : AlsaInfo.txt

apport information

tags: added: apport-collected artful wayland-session
description: updated
Revision history for this message
Peter Milley (pbmilley) wrote : CRDA.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : CurrentDmesg.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : IwConfig.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : JournalErrors.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : Lspci.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : Lsusb.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : ProcModules.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : PulseList.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : RfKill.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : UdevDb.txt

apport information

Revision history for this message
Peter Milley (pbmilley) wrote : WifiSyslog.txt

apport information

affects: compiz-plugins-main (Ubuntu) → linux (Ubuntu)
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Status changed to Confirmed

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
tags: added: zesty
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.15 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15-rc9

Changed in linux (Ubuntu):
importance: Undecided → Medium
status: Confirmed → Incomplete
tags: added: needs-bisect
Revision history for this message
Peter Milley (pbmilley) wrote :

Hi Joseph,

As requested, I booted and tested the 4.15.0-041500rc9-generic kernel, and the problem still exists. Let me know if you have any other questions.

tags: added: kernel-bug-exists-upstream
Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

This issue appears to be an upstream bug, since you tested the latest upstream kernel. Would it be possible for you to open an upstream bug report[0]? That will allow the upstream Developers to examine the issue, and may provide a quicker resolution to the bug.

Please follow the instructions on the wiki page[0]. The first step is to email the appropriate mailing list. If no response is received, then a bug may be opened on bugzilla.kernel.org.

Once this bug is reported upstream, please add the tag: 'kernel-bug-reported-upstream'.

[0] https://wiki.ubuntu.com/Bugs/Upstream/kernel

Changed in linux (Ubuntu):
status: Confirmed → Triaged
Revision history for this message
Peter Milley (pbmilley) wrote :

I've followed the instructions in the Wiki page and sent the email to the maintainers. I will update when I hear back from them.

Revision history for this message
Peter Milley (pbmilley) wrote :

Here is the link to the email sent to the maintainers list: https://marc.info/?l=linux-rtc&m=151683917607842&w=2

tags: added: kernel-bug-reported-upstream
Revision history for this message
Peter Milley (pbmilley) wrote :

Still no response from the kernel team. After a week, I tested the newest daily build of the kernel, and the problem still exists. Sent a follow-up email with that info, and haven't heard anything.

Revision history for this message
Peter Milley (pbmilley) wrote :

Just tested again, and the problem still exists in kernel 4.16.0-041600rc1-generic. I'll be sending that update to the kernel maintainers as well.

Revision history for this message
Kai-Heng Feng (kaihengfeng) wrote :

Can you do a kernel bisection?

First, find the last good -rc kernel and the first bad -rc kernel from http://kernel.ubuntu.com/~kernel-ppa/mainline/

Then,
$ sudo apt build-dep linux
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
$ cd linux
$ git bisect start
$ git bisect good $(the good version you found)
$ git bisect bad $(the bad version found)
$ make localmodconfig
$ make -j`nproc` deb-pkg
Install the newly built kernel.
If the issue still happens,
$ git bisect bad
Otherwise,
$ git bisect good
Repeat to "make -j`nproc` deb-pkg" until you find the commit that causes the regression.

Revision history for this message
Alexandre BELLONI (abelloni) wrote :

The issue is caused by CONFIG_WDAT_WDT which driver is stealing the RTC resources before the RTC driver is probed. Disabling CONFIG_WDAT_WDT will solve that issue.

Revision history for this message
Peter Milley (pbmilley) wrote :

A bug has been filed with the kernel developers here: https://bugzilla.kernel.org/show_bug.cgi?id=199033

Also, I have confirmed that disabling CONFIG_WDAT_WDT and re-compiling the kernel, as Alexandre suggested, does allow the RTC to run properly.

Revision history for this message
Gunter Ohrner (gohrner) wrote :

This also affects my system in Ubuntu 18.04 LTS - very annoying!

Apparently, according to the upstream bug report, a proper fix is included in Linux 4.18 (https://bugzilla.kernel.org/show_bug.cgi?id=199033#c91) - can we expect a backport for Ubuntu LTS?

Revision history for this message
Gunter Ohrner (gohrner) wrote :

Handle 0x0001, DMI type 1, 27 bytes
System Information
        Manufacturer: Sony Corporation
        Product Name: SVS13A3W9ES
        Version: C60BRBTW
        Serial Number: 54528306-0000358
        UUID: E6F5C740-64C8-11E2-A592-3C0771848C83
        Wake-up Type: Power Switch
        SKU Number: 54528306
        Family: VAIO

Changed in linux (Ubuntu):
status: Triaged → Confirmed
Revision history for this message
Lawrence Houston (lhoust) wrote :

Within the 4 computer I have running Ubuntu, only one, my Toshiba Satellite Laptop with the current 4.15 Kernels available from the Ubuntu 16.04.5 and 18.04.1 Repositories fails to create the /dev/rtc device... NOTE: The 3 other Ubuntu Machine are not effected by this CONFIG_WDAT_WDT Bug...

Downgrading to the current Generic 4.4 Kernel from the Ubuntu 16.04.5 Repository is one Solution...

Another Solution is the 4.18 Kernel currently available from Ubuntu the 18.10 Repository!!!

Brad Figg (brad-figg)
tags: added: cscc
Revision history for this message
Adnan Hodzic (fooctrl) wrote :

I have this same problem on Ubuntu 19.10 on ThinkPad X1 Carbon 7th gen.

sudo rtcwake -u -s 60 -m mem
rtcwake: /dev/rtc0: unable to find device: No such file or directory

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.