Linux 5.12 Mostly Restores Long-Horrid AES-XTS Performance Introduced By CPU-Bug Mitigations
AES-NI XTS hardware encryption and decryption performance on Linux has been severely crippled since the Spectre V2 mitigations were introduced to the Linux kernel nearly two years ago. Linux 5.12 has fixes that mostly restore AES-NI XTS performance on the AMD side and those changes slightly increase aex-xts performance on the Intel side.
written by 윤채경 (Yoon Chae-kyung) 2021-03-02 - last edited 2021-03-02. © CC BY
A LUKS-encrypted file. You can tell by the header and fact that the first four bytes say LUKS.
The Spectre V2 security-mitigations that were introduced to the Linux kernel almost two years ago had a dramatic, yet for most people not noticeable, effect on hardware accelerated AES-NI XTS performance on both Intel and AMD hardware. This is highly relevant for everyone who enjoys using full disk encryption using the standard LVM+LUKS setup most distributions that offer full disk encryption default to.
aes-xts performance with a 256 bytes long key was cut down to about two thirds of what it was on AMD-powered machines, and performance with a 512 byte key was nearly cut in half:
| Kernel | Spectre V2 mitigations |
Algorithm | Key | Encryption | Decryption |
|---|---|---|---|---|---|
| 5.9.8 | off | aes-xts | 256b | 3053.6 MiB/s | 3091.4 MiB/s |
| 5.11.1 | off | aes-xts | 256b | 3031.2 MiB/s | 3073.2 MiB/s |
| 5.12rc1 | off | aes-xts | 256b | 3106.3 MiB/s | 3079.5 MiB/s |
| 5.9.8 | on | aes-xts | 256b | 1807.9 MiB/s | 1807.4 MiB/s |
| 5.11.1 | on | aes-xts | 256b | 1813.4 MiB/s | 1798.2 MiB/s |
| 5.12rc1 | on | aes-xts | 256b | 2947.3 MiB/s | 3024.3 MiB/s |
| Kernel | Spectre V2 mitigations |
Algorithm | Key | Encryption | Decryption |
| 5.9.8 | off | aes-xts | 512b | 2685.7 MiB/s | 2706.6 MiB/s |
| 5.11.1 | off | aes-xts | 512b | 2666.0 MiB/s | 2664.9 MiB/s |
| 5.12rc1 | off | aes-xts | 512b | 2596.5 MiB/s | 2566.2 MiB/s |
| 5.9.8 | on | aes-xts | 512b | 1578.9 MiB/s | 1581.6 MiB/s |
| 5.11.1 | on | aes-xts | 512b | 1597.6 MiB/s | 1593.9 MiB/s |
| 5.12rc1 | on | aes-xts | 512b | 2350.0 MiB/s | 2462.4 MiB/s |
There is a pretty big difference between 2685.7 MiB/s and 1578.9 MiB/s, yet the lack of a practical effect is probably why this has gone unnoticed for so long. A SATA3 interface caps out at 600 MiB/s so it doesn't really matter if a full-disk encrypted machine can do hardware accelerated AES-XTS at 601 or 5000 MiB if the disk drive is connected to a SATA interface. It does make a difference if the machine has a bleeding edge high-end NVMe drive, which is likely why this was performance-regression was discovered and fixed. Linux 5.12rc1 performs a whole lot better than previous kernels did when Spectre V2 mitigations are enabled on AMD machines. The difference between the kernels default mitigations and mitigations=off is barely noticeable. That's a huge improvement mostly thanks to the hard work of Google-asset Ard Biesheuvel
The CPU-bug mitigations and the changes in Linux 5.12 have a similar effect on the Intel side:
| Kernel | Spectre V2 mitigations |
Algorithm | Key | Encryption | Decryption |
|---|---|---|---|---|---|
| 5.11.1 | off | aes-xts | 256b | 2049.1 MiB/s | 2050.7 MiB/s |
| 5.12rc1 | off | aes-xts | 256b | 2195.1 MiB/s | 2197.2 MiB/s |
| 5.11.1 | on | aes-xts | 256b | 1548.3 MiB/s | 1550.4 MiB/s |
| 5.12rc1 | on | aes-xts | 256b | 2116.1 MiB/s | 2102.1 MiB/s |
| Kernel | Spectre V2 mitigations |
Algorithm | Key | Encryption | Decryption |
| 5.11.1 | off | aes-xts | 512b | 1602.9 MiB/s | 1602.1 MiB/s |
| 5.12rc1 | off | aes-xts | 512b | 1649.7 MiB/s | 1655.8 MiB/s |
| 5.11.1 | on | aes-xts | 512b | 1249.9 MiB/s | 1243.8 MiB/s |
| 5.12rc1 | on | aes-xts | 512b | 1608.8 MiB/s | 1601.9 MiB/s |
The improvements to the way Linux 5.12 handles hardware accelerated aes-xts encryption is specially impressive on the Intel side. Not only is performance restored, Linux 5.12rc1 is faster with mitigations on than Linux 5.11.1 is with mitigations off. This is probably due to overall aes-xts improvements in Linux 5.12rc1, it is also faster than 5.11.1 when mitigations are disabled on both kernels.
Brand new processors do not need the spectre V2 mitigations, so this makes no difference if you just bought your CPU, but it does if you have a AMD processor up to and including Zen 3 or a Intel processor that is more than a year old.
Linux 5.12 will be released in about eight weeks from now. The first release-candidate can be acquired from kernel.org. You should probably stick with 5.11 or older unless you really want the new aes-xts improvements and you're willing to take a risk, it is not unusually for severe bugs to be found during a kernels release-candidate cycle.
Enable comment auto-refresher