Zram
zram is a Linux kernel module that allows you to create RAM-backed block devices with up to 5:1 compression. zram devices can be used like any other block device. They are typically used to make compressed swap devices using a machines RAM. They can also be used to store /tmp and anything else one might want to store on a compressed RAM-drive.
Fedora 33+ creates a zram swap device using half the systems memory, limited to a maximum of 4 GiB, using a tool called zram-generator.
Contents
Basic Usage
Enabling a zram swap device in Cool-Retro-Term.
The zram module must be loaded to use zram. You can load it with modprobe zram if you want one zram block device or modprobe zram num_devices=4 if you want to create four zram devices when you initialize it. You can also set the number of devices by creating a file in /etc/modprobe.d/:
options zram num_devices=4
Creating Devices
zram devices are created as /dev/zram0, /dev/zram1 and so on. Additional drives can be created by simply reading /sys/class/zram-control/hot_add:
cat /sys/class/zram-control/hot_add
The compression algorithm can be changed by echoing to /sys/block/zram0/comp_algorithm. Reading that file with cat /sys/block/zram0/comp_algorithm outputs a list of supported algorithms you can use with the one in use in brackets. That list could look like lzo lzo-rle lz4 lz4hc 842 [zstd] or just [lzo] lzo-rle depending on where you got your kernel and who compiled it.
The trade-offs of various compression algorithms may not be what you think they are when you use zram. The module assumes that the compression ratio is about 2:1 and acts accordingly. Better compression will result in less actual allocated memory, and how fast the algorithm is will have an impact on performance. You will want to use zstd, lzo-rle or lzo in that order of preference (use the best one available in your kernel).
A zram drives size is 0 until you change that.
A zram devices size can be set by echoing a size to /sys/block/zram{devicenumber}/disksize. You can use multiples of bytes like K (or KiB), M and G:
echo 512M > /sys/block/zram0/disksize
echo 1G > /sys/block/zram1/disksize
zram does not use any actual memory before a device is filled with data. A 2 GiB drive with 5.8 MiB of data compressible down to 443.9K will use a total of 780K, not 2 GiB.
zram devices are initialized and ready to be used once a size is set. They will now be listed in /proc/partitions as if they were regular hard drives.
Turning a zram block device into a RAM-backed compressed swap is easy once the drive it initialized, it is merely a matter of running mkswap to make it a swap drive and swapon to enable that drive. Using the swapon argument -p 32767 will give a zram device the highest priority possible.
mkswap /dev/zram0
swapon /dev/zram0 -p 32767
Removing Devices
You can't just eradicate a zswap device while it is being used. You must first unmount any file system, or swap device, mounted on a zram device. Removing unused zram devices is a simple matter of echoing the device number to /sys/class/zram-control/hot_remove:
swapoff /dev/zram0
echo 0 > /sys/class/zram-control/hot_remove
Examining Devices
util-linux comes with a tool called zramctl which can be used to show a drives true compression ratio and other information. That information is also available in /sys/block/zram{devicenumber}/. You can do a close-up inspection of those files with
grep -H '' /sys/block/zram0/*
Simple Script For Using Zram As A Swap Device
You should use zram-generator (see below) to manage zram swap drives if you are using Fedora 33 or higher.
This simple script will create a zram swap device. Note that it assumes you are not using zram for anything else. It is not very elegant but it works just fine.
#!/bin/bash
# Disable zswap
echo 0 > /sys/module/zswap/parameters/enabled
# Load zram module
modprobe zram
# use zstd compression
echo zstd > /sys/block/zram0/comp_algorithm
# echo 512M > /sys/block/zram0/disksize
echo 2G > /sys/block/zram0/disksize
mkswap /dev/zram0
# Priority can have values between -1 and 32767
swapon /dev/zram0 -p 32767
This short simple script will remove the zram swap device. It, again, assumes that you are only using zram for compressed swap.
#!/bin/bash
swapoff /dev/zram0
echo 0 > /sys/class/zram-control/hot_remove
# Not required, but creating a blank uninitalzed drive
# after removing one may be desired
cat /sys/class/zram-control/hot_add
Remember to make the scripts executable with chmod +x.
This simple systemd unit file lets you have systemd start your zram swap drive at boot:
[Unit]
Description=Configures zram swap device
After=local-fs.target
[Service]
Type=oneshot
ExecStart=/usr/local/bin/zramswap-on
ExecStop=/usr/local/bin/zramswap-off
RemainAfterExit=yes
[Install]
WantedBy = multi-user.target
Create that service file, reload the service files with systemctl daemon-reload and you can enable and start it with
systemctl enable --now create-zram-swap.service
Compression Algorithms
Chromium 85 being compiled in a QEMU virtual machine with 6 GiB RAM and a 3 GiB zstd zram compressed swap.
Comparing how compression algorithms play out when they are used to store files on a a zram-backed regular file system in a repeatable fashion is easy. Here's how much RAM it takes to store the Linux kernel on a zram device:
| Algorithm | cp time | Data | Compressed | Total |
|---|---|---|---|---|
| lzo | 4.571s | 1.1G | 387.8M | 409.8M |
| lzo-rle | 4.471s | 1.1G | 388M | 410M |
| lz4 | 4.467s | 1.1G | 403.4M | 426.4M |
| lz4hc | 14.584s | 1.1G | 362.8M | 383.2M |
| 842 | 22.574s | 1.1G | 538.6M | 570.5M |
| zstd | 7.897s | 1.1G | 285.3M | 298.8M |
See Comparison of Compression Algorithms: zram block drive compression for details on how the above tests were done.
Comparing how the different compression algorithms play out when a zram block device is used as a swap device is a very different matter. This is how much memory is used when a 3 GiB zram swap is fully filled while building Chromium 85 in a virtual machine with just 6 GiB total system memory available (and a HDD backed swap):
| Algorithm | Disk size | Data | Compressed | Total memory used |
|---|---|---|---|---|
| zstd | 3G | 3G | 565.8M | 599.4M |
| lzo-rle | 3G | 3G | 811.1M | 851M |
The above real-world zram swap compression number with zstd indicate a 5:1 compression ratio while lzo-rle does something in the neighborhood of 3.7:1. This may or may not reflect what you get in your particular workload.
Performance Implications
Compiling Chromium with four threads on an old Athlon 5350 APU with a slow HDD as a swap device in addition to a zram swap device indicate that a small zram swap can have a negative impact while a larger zram swap has a positive impact. The slow old Athlon 5350 APU had two DDR3 1600 4 GiB sticks but only 7 GiB available due to 1 GiB being reserved by the APU.
| RAM | zram swap | zswap (cache) | compression | time | difference |
|---|---|---|---|---|---|
| 7 GiB | none | none | none | 29h 9m (1748m48.292s) | Baseline |
| 7 GiB | GiB | none | lzo-rle | 29h 14m 1753m57.703s | +5 minutes |
| 7 GiB | 1 GiB | none | zstd | 30h 23m (1823m56.761s) | +1 hours 15 minutes |
| 7 GiB | 2 GiB | none | zstd | 28h 46m (1725m45.160s) | -23 minutes |
| 7 GiB | 3.5 GiB (50% of physical RAM) |
none | lzo-rle | 27h 25m (1645m13.886s) | -1 hour 44 minutes |
| 7 GiB | 3.5 GiB (50% of physical RAM) |
none | zstd | 27h 28m (1647m58.215s) | -1 hour 41 minutes |
| 7 GiB | none | 10% (700 MiB) zbud |
zstd | 29h m4hm (1782m4.096s) | + 33 minutes |
| 7 GiB | none | 10% (700 MiB) z3fold |
zstd | 30h 18m (1818m52.867s) | +1 hour 10 minutes |
| 7 GiB | none | 10% (700 MiB) zsmalloc |
zstd | 28h 43m (1723m9.704s) | -26 minutes |
| 7 GiB | none | 20% (1400 MiB) z3fold |
zstd | 30h 18m (1818m55.137s) | +1 hour 10 minutes |
The result are very different in a virtual machine with an artificial memory constraint. The performance impact is exact opposite of what it is in the real hardware test shown above: A larger zram swap device performs worse than a smaller one. This may be cause the non-zram swap device in the test below isn't entirely real: Host system caching is likely skewing the numbers below in ways that make a notable difference.
| Compiling Chromium requires a lot more than 8 GiB RAM. Compiling it with less causes a lot of swap activity making it an ideal test for zram and zswap efficiency. | ||||||
| RAM | zram swap | Compression | Disk swap | zswap cache | Compile time | Benefit |
|---|---|---|---|---|---|---|
| 20G | None | 20G | None | 4h 30m (270m) | ||
| 8G | None | 20G | None | 5h 33m (333m) | Baseline | |
| 8G | 4G | None | None | FAIL (OOM) | ||
| 8G | 1G | zstd | 20G | None | 5h 26m (326m) | -7m |
| 8G | 2G | zstd | 20G | None | 5h 19m (319m) | -14m |
| 8G | 2G | lzo-rle | 20G | None | 5h 24m (324) | -9m |
| 8G | 4G | zstd | 20G | None | 6h 21m (381m) | +48m |
| 8G | 4G | lzo-rle | 20G | None | 6h 3m (363m) | +30m |
| 8G | None | zstd (z3fold) | 20G | 4% | 5h 32m (332m) | -1m |
| 8G | None | zstd (z3fold) | 20G | 10% | 6h (360m) | +27m |
| Note how the enough RAM scenario is better than any combination of 8 GiB RAM and zram swap or zswap cahce. | ||||||
Utilities
zramctl
zramctl, part of the util-linux package, can be used to setup and configure zram devices. Running zramctl with no arguments makes it output a list of zram devices and their status:
$ zramctl
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram1 zstd 512M 0B 0B 0B 12
/dev/zram0 zstd 2G 4K 58B 4K 12 [SWAP]
zramctl has a -f or --find option that makes it either identify and use the first unused zram device or create a new zram device is no used devices are unavailable. That option can be used together with -s or --size and optionally -a or --algorithm to initialize a zram device:
zramctl --find --size 512M --algorithm zstd
zramctl will echo what device it played with when that command is executed. That would be /dev/zram0 if no devices are in use or /dev/zram2 if there are two existing zram devices present. That can be used as part of a script:
newdrive=`zramctl --find --size 512M --algorithm zstd`
echo ${newdrive}
See the zramctl.8 manpage for all the options. Note that the options the manual lists for --algorithm may not reflect what is available on your system, cat /sys/block/zram0/comp_algorithm to see what is actually available on your machine.
zram-generator
Fedora 33+ comes with, and by default, activates a tool called zram-generator (available from github.com /systemd/zram-generator).
zram-generator will, left unconfigured and ignored, create a zram swap device using half the systems memory but no more than 4 GiB in total (it will create a 2 GiB zram swap device if you have 4 GiB total, but only a 4 GiB zram swap device if you have 128 GiB of memory). It defaults to lzo-rle compression. That behavior is dictated by the package zram-generator-defaults which only contains a configuration file named /usr/lib/systemd/zram-generator.conf. You can override the default settings by creating a configuration file named /etc/systemd/zram-generator.conf:
[zram0]
# Use 20% of system memory for zswap. Default: 0.5
zram-fraction = 0.2
# Limits the maximum size. Set in MiB. 2048 means
# it will never be larger than 2 GiB. Default: 4096
max-zram-size = 2048
# Fedora defaults to lzo-rle which is very inefficient
# compared to vastly superior zstd compression
compression-algorithm = zstd
zram-generator gained support for setting a swap priority on the zram swap devices it creates on January 13th, 2021. The default value, set by /src/config.rs, is 100.
swap-priority=32767
The swap-priority= setting, available in v0.3.0-rc.1 and newer, is likely something you want if you have existing swap devices. A pretty typical disk swap priority of 5000 or 10000 (the range is >-1 to 32767 where 32767 has highest priority) will make Linux swap to disk before it swaps to the RAM-backed swap device if the foolish in-practice lowest-priority zram-generator default of just 100 is used.
zram-generator can be disabled using any of three different methods:
- mask it by running
systemctl mask swap-create@.service(that's the actual service you need to mask, it is not namedzram-generator) - create a _blank_ configuration file (
echo > /etc/systemd/zram-generator.conf) - eradicate the
zram-generatorandzram-generator-defaultspackages (dnf -y remove zram-generator zram-generator-defaults).
A zram swap is a good addition to a regular swap device as long as the drives size is less than 25% of total system memory so you may want to configure it correctly instead of disabling it if you are unhappy with the silly Fedora 33 defaults.
zram and zswap
The Linux kernel has a similarly-sounding kernel module called zswap. It is easy to confuse a zram swap with a zswap since it sounds similar. However, they are not the same thing.
zswap is fundamentally different in one simple way: It is a swap drive cache. It can compress pages going to a swap device in RAM and compress up to 3:1 if you use zstd compression and the z3fold zpool.
Pick one or the other, don't use both. Using a compressed memory cache to cache a compressed memory block device is foolish unless you have some very unusual and very specific requirements. Using it to cache a hard disk swap does make sense. You can not configure zswap to only cache a specific swap device so it does not sense to use it even if your machine has a real swap if you also use a zram swap.
See Also
- zswap can be used to create a up to 3:1 memory compressed swap cache.
Enable comment auto-refresher