Linux Kernel Disables Coffee Lakes HPET On The Grounds That It Is "Unreliable"
Linus Torvalds has pulled a "fix" which disables the High Precision Event Timer on Intel Coffee Lake systems into the git mainline kernel on the grounds that it's "unreliable". You may want to use the nohpet kernel parameter on Coffee Lake and Ice Lake (same problem, not fixed) machines. Those platforms HPET's problems with keeping track of time in low-power states have cast light on some serious not-fixed problems with kernel's clocksource watchdog.
A fix which disables HPET on Intel Coffee Lake systems has been pulled into the current git mainline kernel which will eventually become 5.4.
The High Precision Event Timer was introduced back in 2005 to aid multimedia playback. It is costly and inefficient to use as a timer source. The Linux kernel will typically try to validate that the processors Time Stamp Counter, which counts the number of cycles since reset, functions properly and use that as it's clock-source. The kernel ring buffer, which can be seen by using the dmesg command, will typically contain a line saying clocksource: Switched to clocksource tsc.
Coffee Lake and Ice Lake systems have a HPET which is unreliable when the system goes into an low-power idle state. Leaving the machine alone with no load for a moment is enough to make it stop counting. That is a defect of sorts but it should not be a problem. Yet it is.
It turns out that the kernel's timer watchdog will use the HPET to verify that the TSC is a working clock-source and switch to HPET as a clock source if the HPET is broken. Kernel Bug 203183 has quite a few descriptions of what happens when a Coffee lake machine goes into an idle state and it's HPET stops functioning:
- clocksource: timekeeping watchdog on CPU7: Marking clocksource 'tsc' as unstable because the skew is too large:
- clocksource: 'hpet' wd_now: 54b0f25 wd_last: 4e9fe44 mask: ffffffff
- clocksource: 'tsc' cs_now: 3e0e2debe0 cs_last: 3daab92440 mask: ffffffffffffffff
- tsc: Marking TSC unstable due to clocksource watchdog
- TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'.
- sched_clock: Marking unstable (4024083039, -219)<-(4075977403, -51894628)
- clocksource: Switched to clocksource hpet
Linus Torvalds just merged commit fe30021c36fbfb71d6 which "solves" this by disabling the HPET on Intel Coffee Lake systems (0x3ec4) in arch/x86/kernel/early-quirks.c. The line disabling that HPET was added below a similar line which disables the HPET on the Intel Baytrail (0x0f00) platform (for other reasons).
The Intel "Ice Lake" platform was not blacklisted even though it suffers the exact same issue. Those who own Ice Lake systems can use the nohpet kernel parameter for now. nohpet can also be used by Coffee Lake owners who run current stable kernels which do not have the commit which disables the HPET on Coffee Lake systems.
Kernel Bug 203183 mentions that affected machines who do end up using HPET as the clocksource have a clock drift of about 3 seconds per hour. This is not noticeable if the system is setup to synchronize the clock over the network using the Network Time Protocol but it does add up if there is no regular clock adjustment against external sources.
It's interesting to note that no patches or proposals have been made to solve the way the Linux kernel's clocksource watchdog decides what clocksource is "reliable". Coffee Lake's broken HPET reveals that if the HPET clocksource is drifting and the TSC is fine the kernel will decide that using the HPET is the best option. Disabling the HPET on a per-platform basis seems like a unwise band-aid solution. Perhaps some kernel-developer should look into a wiser solution on a rainy day.
published 2019-11-17 - last edited 2019-11-17
Enable comment auto-refresher