HOWTO make Linux run blazing fast (again) on Intel CPUs

It's just been one security disaster after another for Intel the last few years. Meltdown, Spectre variant after variant and this week the "Microarchitectural Data Sampling" aka Zombieload attack have all required performance-degrading fixes and workarounds. There is no way around turning hyperthreading off to be safe from MDS/Zombieload and this is a rather high performance-price to pay. So what if you don't want to?

Disabling SMT/HyperThreading to get full protection against MDS/Zombieload on top of the mitigation code for "meltdown", several "spectre" variants and other security-issues discovered on Intel CPUs is a high price to pay for security on Intel CPUs. The total performance-penalty in many workloads is adding up. Unfortunately there is no safe and secure way around the performance-penalties - so you may want to..

TAKE THE RISK?

If you're not into currency trading or high finance or military contracting or anything of that nature and you'd just like to get maximum performance for your Steam games then adding this is simple switch to your kernel parameters will leave you wide open to all the security risks for maximum excitement and squeeze back every bit of performance you used to get from your Intel CPU:

mitigations=off

If you are using a kernel older than 5.1.13 then you should use this rather long one-liner instead:

noibrs noibpb nopti nospectre_v2 nospectre_v1 l1tf=off nospec_store_bypass_disable no_stf_barrier mds=off mitigations=off

Add either mitigations=off or that long one-liner to your /etc/sysconfig/grub and re-generate grub's configuration file with grub2-mkconfig (your distributions procedure will vary) and you're all set. Do note that the latest editions of stable branch kernels (4.14.x, 4.19.x) do have a mitigations= parameter so that alone is enough on kernels newer than 5.1.13 and later versions of stable branch kernels such as 4.19.60+.

Debian/Ubuntu derived distributions: edit the file /etc/default/grub then run the commands update-grub followed by grub-install /dev/sdX where "X" is replaced by the relevant OS drive, usually "a" as in "/dev/sda".

Here is what the above kernel command options did in earlier kernels, one by one:

noibrs - We don't need no restricted indirect branch speculation
noibpb - We don't need no indirect branch prediction barrier either
nospectre_v1 and nospectre_v2: Don't care if some program can get data from some other program when it shouldn't
l1tf=off - Why would we be flushing the L1 cache, we might need that data. So what if anyone can get at it.
nospec_store_bypass_disable - Of course we want to use, not bypass, the stored data
no_stf_barrier - We don't need no barriers between software, they could be friends
mds=off - Zombieload attacks are fine
mitigations=off - Of course we don't want no mitigations

You are (probably) an adult. You can and should wisely decide just how much risk you are willing to take. Do or don't try this at home. You do not want to try this at work.

Note: The parameters covered by mitigations=off vary by kernel versions. The above switches were applicable prior to kernel 5.1.13 and while they are useful for kernels released prior to the mitigations=off switch they are not current.

You can look at the file Documentation/admin-guide/kernel-parameters.txt in the kernel source for the kernel you are using to see what parameters are actually available on the kernel you are using.

Security parameters covered by mitigations=off in newer kernels

Intel CPUs are not alone in having some security issues. There are problems with other CPUs too. The mitigations=off can be used on any CPU but what it does, if anything, will depend on what CPU you are using. It can be used to slightly increase performance on Intel, AMD, ARM and even PowerPC architectures.

The security parameters covered by mitigations=off in kernel 5.3.6 are:

nopti [X86,PPC] - Control Page Table Isolation of user and kernel address spaces. Disabling this feature removes hardening, but improves performance of system calls and interrupts.
kpti=0 [ARM64] - Control page table isolation of user and kernel address spaces.
nobp=0 [S390] - Undocumented. Does something on S390 systems, nobody knows what.
nospectre_v1 [X86,PPC] - Disable mitigations for Spectre Variant 1 (bounds check bypass). With this option data leaks are possible in the system.
nospectre_v2 [X86,PPC,S390,ARM64] - Disable all mitigations for the Spectre variant 2 (indirect branch prediction) vulnerability. System may allow data leaks with this option.
spectre_v2_user=off [X86] - Control mitigation of Spectre variant 2 (indirect branch speculation) vulnerability between user space tasks
spec_store_bypass_disable=off [X86,PPC] - Control Speculative Store Bypass (SSB) Disable mitigation (Speculative Store Bypass vulnerability)
ssbd=force-off [ARM64] - Speculative Store Bypass Disable control
l1tf=off [X86] - Control mitigation of the L1TF vulnerability on affected CPUs
mds=off [X86] - Control mitigation for the Micro-architectural Data Sampling (MDS) vulnerability.

HOWTO check mitigations are enabled or not

This fine command will print out a list of possible vulnerabilities and their mitigation status using fgrep:

fgrep -r '' /sys/devices/system/cpu/vulnerabilities

You can get a identical-looking list using grep -H '':

grep -H '' /sys/devices/system/cpu/vulnerabilities/*

You can also get such a list with awk:

awk 'FNR==1{print "==>"FILENAME"\n"}1' /sys/devices/system/cpu/vulnerabilities/*

It can also be done by creating a function that prints a filename followed by its contents and running it:

header-cat () { local f; for f in "$@" ; do printf '=== %s ===\n' "$f"; cat "$f"; done; }

header-cat /sys/devices/system/cpu/vulnerabilities/*

You can get a similar output from the less and possibly least elegant:

for f in /sys/devices/system/cpu/vulnerabilities/*;do echo $f;cat $f;done

The files in /sys/devices/system/cpu/vulnerabilities/ will either contain Mitigation (with a comment about how that is done), Vulnerable (if mitigations are disabled) or Not affected.

Performance implications

The performance gained by disabling workarounds for the mostly Intel CPU security flaws are not all that impressive in all workloads. The reason is that the most performance-hampering measure required to safely use a Intel CPU is to disable SMT (HyperThreading). Doing so crushes performance in a really noticeable way, so much so that the Linux kernel developers decided to leave SMT enabled by default (unlike some *BSD variants who do disable SMT). Newer Linux kernels will by default use mds=full and not the safer mds=full,nosmt parameter which banks and financial institutions should be using. There is a difference between default performance and performance with mitigations=off but it is nowhere near as large as the difference between mitigations=off and mds=full,nosmt.

As the above charts show: The effect of default parameters vs mitigations=off is measurable but not hugely impressive. The effect of disabling SMT, which is required to safely use Intel CPUs, is really noticeable and very measurable. It is required to be sure Intel CPU bugs can not be exploited but the price is high. Choose wisely.

published 2019-05-17 - last edited 2020-07-08

4.00

(one vote)

Enable comment auto-refresher

Anonymous (4fdc6c74a5)

43 months ago

Score 0

thank you!

Permalink | Reply

Anonymous (43d6f23791)

41 months ago

Score 0

total blast

Permalink | Reply

Anonymous (386f1b50c8)

32 months ago

Score 0

For the curious, an explicit list of the mitigations currently (5.18-rc3) covered by that meta-switch can be found in Documentation/admin-guide/kernel-parameters.txt (the last four have been added since 5.3.6 list):

mitigations=off Equivalent to: nopti [X86,PPC] kpti=0 [ARM64] nospectre_v1 [X86,PPC] nobp=0 [S390] nospectre_v2 [X86,PPC,S390,ARM64] spectre_v2_user=off [X86] spec_store_bypass_disable=off [X86,PPC] ssbd=force-off [ARM64] l1tf=off [X86] mds=off [X86] tsx_async_abort=off [X86] kvm.nx_huge_pages=off [X86] no_entry_flush [PPC]

no_uaccess_flush [PPC]

Permalink | Reply