Linux Kernel Runtime Guard 0.9.1 Is Released
The Linux Kernel Runtime Guard (LKRG) is a Linux kernel security module developed by Openwall for security enthusiasts, Internet-facing production servers and hosting providers that provides additional run-time integrity and security checks for the Linux kernel. The latest version adds support for CONFIG_HAVE_STATIC_CALL on Linux 5.10+ and a fix for a false positive bug on machines with SELinux enabled, and make install no longer enables it by default. There is, instead, a message explaining how to start and/or enable it using the systemd service file it provides.
written by 윤채경 (Yoon Chae-kyung) 2021-04-28 - last edited 2021-05-09. © CC BY
src/modules/kmod/p_kmod.c from LKRG 0.9.1
The Linux Kernel Runtime Guard (LKRG) is an out-of-tree kernel module from Openwall you can install and load as a kernel module or, since version 0.9.0, build into your Linux kernel. LKRG does run-time integrity checks to detect security vulnerability exploits against the Linux kernel.
LKRG is not for everyone and the developers do not recommend it for casual Linux desktop users.
Who is the Kernel Runtime Guard suitable for?
"Shared/cloud hosting providers and their users. Security enthusiasts. Security-hardened distros."
Do you recommend it for casual Linux desktop users?
"Generally, no. There isn't much local security on typical Linux desktop systems/distros anyway, in the way those systems are configured and used, irrespective of any locally exploitable Linux kernel vulnerabilities. Thus, adding LKRG would result in inconsistent security. Specifically, on a typical single-user Linux desktop system there's that one user account that uses "sudo" to access root, so it is effectively root-equivalent (only different from root in terms of safety from mistakes, not really in terms of security). LKRG does provide some relevant security hardening to such systems anyway - it might detect a web browser vulnerability exploit trying to escape the web browser's sandbox - so it could help e.g. a vulnerable system withstand web browser attacks in the Pwn2Own contest. However, that's not how most end-user systems are actually compromised (when they are).
A security enthusiast can very well reasonably use LKRG on their desktop system, though, primarily for consistency with their other systems, and for advance testing of kernel+LKRG combinations prior to deployment on the servers the person manages."
Do you recommend it for and all kinds Internet-facing servers?
"Not necessarily "all kinds", but generally yes. LKRG helps on systems where there are multiple users (or at least pseudo-users for multiple services), and especially where kernels might not be updated or rebooted into in a timely manner after new vulnerabilities are discovered."
So it's not just for firewall deployments?
"No, it is primarily for servers that provide much more than firewall functionality. It isn't nearly as useful on firewalls - not much need for local security when the device does just one thing anyway. However, with web-managed firewalls and such this gets nuanced - the web server is a separate service, hopefully running in its privilege-separated context from the actual firewall functionality (wishful thinking?) and LKRG can help harden that separation."
LKRG supports four different levels of security. The two lowest profiles will log security violations and the two highest will cause a kernel panic if a violation is detected. That leaves a machine hanging if a exploit attempt is unless you boot with the kernel parameter kernel.panic=30 (reboots after 30 seconds) or add panic=60 to a file in /etc/sysctl.d/.
LKRG 0.9.1 is a minor bug-fix release for the bigger 0.9.0 release earlier this month:
"The following major changes have been made between LKRG 0.9.0 and 0.9.1:
- Support CONFIG_HAVE_STATIC_CALL on Linux 5.10+
- Fix SELinux integrity violation false positive bug (introduced into LKRG in March 2021 and manifesting itself on Linux 4.17+ when SELinux is already in enforcing mode when LKRG is loaded)
- Improve systemd service and its installation, add /etc/sysctl.d/lkrg.conf
- Add the debian/ directory in order to support the Debian build system based on pbuilder/dpkg-buildpackage"
LKRG 0.9.0 caused a immediate panic when the p_lkrg was loaded on machines with SELinux enabled. That is no longer the case, 0.9.1 works fine SELinux. A lot of effort has been put into making LKRG play well with SELinux, so it was a bit unfortunate that a regression prior to the 0.9.0 release made it look like it wasn't supported. Well, it wasn't since it didn't work, but it was supposed to and now it does.
We are not entirely sure what the new CONFIG_HAVE_STATIC_CALL support entails. LKRG 0.9.0 appeared to work fine with kernels compiled with CONFIG_HAVE_STATIC_CALL=y. The release notes do not indicate if additional security checks were added specifically for it.
LKRG no longer enables itself upon installation with make install when it is compiled from source. make install will, instead, show instructions for enabling it with the systemd service file it provides. You will now have to manually start it with systemctl start lkrg.service or systemctl enable --now lkrg.service if you want to start and enable it as a boot-up service. You may want to start LKRG and run it for a while to see if there are any false positives or other issues before you enable it as a boot-up service.
The other changes are specific to the Debian build system.
21 commits were added since LKRG 0.9.0, by 5 contributors.
This breakdown may be slightly misleading due to the differences in commit separation, one large commit could have more lines of code than 10 small commits put together.
The bigger changes to the Linux Kernel Runtime Guard came with version 0.9.0 released earlier this month. It added support for building LKRG into a Linux kernel using a script bundled in scripts/copy-builtin.sh. The script assumes you have the kernel source in /usr/src/linux so you will need to either have it there or modify the script. Adding support LKRG to the kernel tree makes a new SECURITY_LKRG option. available. You can still choose to build it as a module with SECURITY_LKRG=m if you make it a part of your kernel tree.
The execve hooks in the LKRG code were replaced with security_ptrace_access and security_bprm_committing_creds and validation of wake-up tasks were removed when LKRG 0.9.0 was released.
LKRG 0.9.0 also made it clear that it does not support real-time kernels with a clean error message on build attempts. Real-time kernels were never supported, 0.9.0 just made that clearer to anyone trying to use it with real-time kernels.
"RT kernels are not supported primarily because their use cases and priorities are inconsistent with LKRG's - it's safety vs. security. Usually safety and security go hand in hand, but in the case of LKRG it's a tradeoff. Besides, this would be more work for us, and yes there are also performance considerations, but these concerns are secondary."
in an e-mail on April 26th, 2021
Performance[edit]
Extracting and compiling the Linux kernel with LKRG 0.9.1 gave a small performance penalty on a Intel-powered laptop and almost none (1.1 seconds) on a AMD Ryzen 2600 powered machine.
The numbers work out to a 1.23% performance reduction on the Intel machine when LKRG is enabled and 0.23% on the AMD machine. That 1.1 second difference on the AMD machine is consistent between runs, but it is so small it hardly matters. The performance penalties will differ in other workloads, so these numbers should be taken with a grain of salt.
Pure CPU workloads like Monero currency mining with XMRig appear to be almost completely unaffected by the LKRG module.
The 0.04% performance loss mining Monero and 0.12% loss mining Wownero seems insignificant, though the day to day crypto currency prices and your electricity costs may vary. The XMRig numbers were made using the PTS test suite which seems to run XMRig with some odd parameters because that box manages about 4kH with --cpu-affinity 0xFF -t6. It really doesn't matter since the runs with and without LKRG were done using the same XMRig options.
The Hard Questions[edit]
We asked the Openwall Linux Kernel Runtime Guard develope team several hard questions about it following the big 0.9.0 release earlier this month.
Why did you write the Kernel Runtime Guard (LKRG)?
"The idea of writing something like LKRG had been growing in my mind for quite a long time. The first serious attempts to write something similar date back to 2009 and my work at CERN. However, the decision to write the first line of LKRG code with its current functionality (more advanced compared to 2009) I made in 2015. The first version of LKRG (0.0) was released in February 2018 under Openwall umbrella.
As you can see, I was struggling with myself as to whether there was a need for such project. It took me some time to convince myself that it's something important and needed. The answer to your question "why did I write LKRG" is the same as what arguments convinced me to pull the trigger and create LKRG. There are several of them:
- I've been an Offensive Security Engineer focusing on rootkits and exploits for some time (since 2003/2004). I've realized that in principle many of the techniques of "infecting" the Linux kernel did not change at all for the last 20+ years. The same applies to many kernel exploitation techniques.
- Other OSes have been facing similar problems as well. However, they tried to address them and some of the solutions have been developed. Those are not perfect, there are some ways to bypass them (although it's not cheap), but it's better than being open to the same 20+ years old techniques.
- Over the years, I've learned that even if some of the protections are imperfect, they can significantly hurt reliability of exploitation or make it harder. In most cases this is good enough to stop mass attacks - those not targeting specific users - and thus protect the users.
- I've also learned that most of the time people who write rootkits or exploits are not the same people who use them. If during an attack the latter people face non-trivial protection or an exploit is not reliable enough (e.g., due to LKRG), they won't be able to or won't bother to adjust the tools and will move on to another target.
- However, even if they are able and willing to adjust the tools, it might be already too late since LKRG will take appropriate action (e.g., panic the kernel), which will be visible to the owner of the system.
It is worth adding that as far as I'm aware, all of the anti-exploitation / security technologies (including mitigations) follow the same path and are generally by-passable, e.g.:
- Firewalls are commonly trivial to bypass, but you still use them
- Antiviruses are trivial to bypass, but people pay money for them
- Mitigations:
- ASLR and especially KASLR are easy to bypass, but people still want to have them enabled in the system
- Non-executable memory is easy to bypass, but people still want to have it enabled in the system
- SMEP is easy to bypass, but people still want to have it enabled in the system
- Stack cookies are bypassable but people still compile binaries with them
- RelRO are bypassable but people still enable that flag
- Any anti-threat technology (e.g., Capsule8, Crowdstrike, WDATP, and many others) can be bypassed or completely disabled (some of them run in user mode), but people pay a lot of money to use them
LKRG is no different here, except for the "price" part, since many of these technologies are not available for free.
Taking all these into account, I came to the conclusion that LKRG can improve Linux ecosystem and bring useful security functionalities that everyone can benefit from. It will also bring some of the "comparable functionality" from other OSes into the Linux world, like Patch Guard, Advanced Threat Protection (ATP), monitoring, and more:
Who are the primary users of the Kernel Runtime Guard (LKRG)?
"As far as we're aware, currently it's security enthusiasts and security hardened distros. We also see interest from security hardened mobile distros (for smartphones and such), but we're not aware of specific uses in those yet (there might be some)."
What are the advantages over plain SELinux or other security solutions?
"Apples to oranges. These are orthogonal, and can reasonably co-exist.
SELinux is a Linux kernel mechanism to enforce policies in the userspace, assuming that the kernel itself is not compromised. SELinux generally doesn't protect the kernel from getting compromised via kernel vulnerabilities, and is generally completely ineffective afterwards. Further, even in absence of Linux kernel vulnerabilities, SELinux is only effective as far as its policies configured on the system are - thus, it generally won't protect a service it's unaware of (has no policy written for).
LKRG is a mechanism to protect the Linux kernel itself, in case the kernel is vulnerable and is (almost) compromised by an attack on it. LKRG is a last-resort defense for the kernel, trying to catch attacks on the kernel that have almost succeeded."
How long will it remain free software?
"We don't intend to ever make LKRG non-free only. We might (or might not) introduce a non-free version, but if we do we intend to also continue with the free software versions."
Adam 'pi3' Zabrocki}}
You can acquire the latest Linux Kernel Runtime Guard 0.9.1 release and read more about it at www.openwall.com/lkrg/. You really should look at the README file if you want to try it, there are a lot of configuration options and some default behavior you may not like. For one, it defaults to either killing specific tasks or causing a kernel panic when a vulnerability exploitation is detected. A kernel panic would leave a remote server hanging when a intrusion, or false positive being seen as a intrusion, is detected. You can easily fix that by adding kernel.panic=60 to the kernel command line so it reboots 60 seconds after a kernel panic or, alternatively, configure LKRG to just log intrusion attempts and not cause kernel panics. Both solutions are simple if you know, and you will learn what you need to know about LKRG if you read the README file within the release tarball.
Enable comment auto-refresher