Huawei is moving to promote their EROFS read-only filesystem from the kernel staging area
Huawei's read-only filesystem EROFS has been stuck in the kernel staging area for almost a year. It is in many ways similar to Squashfs; it's function is to provide a efficient space-saving read-only filesystem. Developer Gao Xiang feels that 10+ million Huawei phone installations is enough to demonstrate that EROFS is mature enough to be granted a proper place in the mainline kernel. We failed testing the Chinese technology.
Huawei asset Gao Xiang sent the following request to the Linux Kernel Mailing List on July 22nd, 2019:
"EROFS file system has been in Linux-staging for about a year. It has been proved to be stable enough to move out of staging by 10+ millions of HUAWEI Android mobile phones on the market from EMUI 9.0.1, and it was promoted as one of the key features of EMUI 9.1, including P30(pro).
EROFS is a read-only file system designed to save extra storage space with guaranteed end-to-end performance by applying fixed-size output compression, inplace I/O and decompression inplace technologies to Linux filesystem.
In our observation, EROFS is one of the fastest Linux compression filesystem using buffered I/O in the world. It will support direct I/O in the future if needed. EROFS even has better read performance in a large CR range compared with generic uncompressed file systems with proper CPU-storage combination, which is a reason why EROFS can be landed to speed up mobile phone performance, and which can be probably used for other use cases such as LiveCD and Docker image as well.
Currently EROFS supports 4k LZ4 fixed-size output compression since LZ4 is the fastest widely-used decompression solution in the world and 4k leads to unnoticable read amplification for the worst case. More compression algorithms and cluster sizes could be added later, which depends on the real requirement."
EROFS has been available in the Linux kernels "staging" for almost a year and it is possible to try it using any mainline kernel as long as you compile with
CONFIG_EROFS_FS enabled. No modern desktop distribution has EROFS enabled, nor do the repositories have the required mkfs.erofs tool. You will have to compile.
Testing the Chinese technology
What you need to try the technology is a) a kernel compiled with
CONFIG_EROFS_FS and b)
mkfs.erofs which is available at git.kernel.org : xiang/erofs-utils.git.
The newly released Linux Kernel 5.3rc1 will give best results since it supports decompression in-place. Older kernels can also be used with the latest user-space utility by adding the "legacy-compress" option.
"I think you should use 5.3-rc1 which is just released since it supports decompression in-place.
you could also use < 5.3 with "mkfs.erofs -E legacy-compress -zlz4hc testfile.erofs testfolder" to generate legacy images (however, it is not perferred though.)
When testing erofs performance, it is better to disable intel_pstate (Turbo boost) and lock the related CPU frequency."
Our first attempt at compiling erofs-utils with lz4 1.8.3 included in the distribution resulted in a
mkfs.erofs which would immediately abort with
mkfs.erofs: compressor.c:74: erofs_compressor_exit: Assertion `!(!c->alg)' failed.. The erofs-utils README and helpful Huawei developers explained that there is a problem with lz4 <=1.8.3 and that a newer LZ4 is desired. One can be acquired like this:
git clone https://github.com/lz4/lz4.git cd lz4/ make su -c 'make install'
Compiling the user-space tool is then strait forward:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b dev cd erofs-utils/ ./autogen.sh ./configure --with-lz4-incdir=/usr/local/include/ --with-lz4-libdir=/usr/local/lib make
You can optionally
su -c 'make install'. The
mkfs.erofs will be in the
mkfs/ folder. There is no manual page and there is also no
--help. Just running
mkfs.erofs reveals the following:
- mkfs.erofs 0.1
- usage: [options] FILE DIRECTORY
- Generate erofs image from DIRECTORY to FILE, and [options] are:
- -zX[,Y] X=compressor (Y=compression level, optional)
- -d# set output message level to # (maximum 9)
- -EX[,...] X=extended options
The syntax for creating a simple image is:
mkfs.erofs -zlz4hc myimage.erofs.img folder1/ folder2/
This works but we did ran into a slight problem.
Testing on a very small folder containing a fork of the it87 sensor driver:
mkfs.erofs -zlz4hc it87.erofs.img it87
resulted in a working
it87.erofs.img image which could be mounted with
su -c 'mount -o loop it87.erofs.img /mnt/tmp'
So far so good.
However, trying something more interesting like compressing the compiled Linux kernel - which is much larger than just the source - was not as easy.
mkfs.erofs -zlz4hc linux-5.3-rc1.erofs.img linux-5.3-rc1/
produced the following complaint:
mkfs.erofs: inode.c:248: erofs_write_dir_file: Assertion `!(used != dir->i_size % (1U << (12)))' failed.
Aborted (core dumped)
What did they mean by this? We asked
gdb. The GNU Debugger has this to say about this issue:
gdb /usr/local/bin/mkfs.erofs (..) (gdb) run -zlz4hc linux-5.3-rc1.erofs.img linux-5.3-rc1/ Starting program: /usr/local/bin/mkfs.erofs -zlz4hc linux-5.3-rc1.erofs.img linux-5.3-rc1/ warning: Loadable section ".note.gnu.property" outside of ELF segments mkfs.erofs 0.1 c_version: [ 0.1] c_dbg_lvl: [ 0] c_dry_run: [ 0] mkfs.erofs: inode.c:248: erofs_write_dir_file: Assertion `!(used != dir->i_size % (1U << (12)))' failed. Program received signal SIGABRT, Aborted. __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 50 return ret; (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007ffff7dbf895 in __GI_abort () at abort.c:79 #2 0x00007ffff7dbf769 in __assert_fail_base (fmt=0x7ffff7f276b8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x407788 "!(used != dir->i_size % (1U << (12)))", file=0x40772d "inode.c", line=248, function=<optimized out>) at assert.c:92 #3 0x00007ffff7dcd566 in __GI___assert_fail (assertion=0x407788 "!(used != dir->i_size % (1U << (12)))", file=0x40772d "inode.c", line=248, function=0x4078c0 <__PRETTY_FUNCTION__.5621> "erofs_write_dir_file") at assert.c:101 #4 0x00000000004032b9 in erofs_write_dir_file () #5 0x0000000000403d38 in erofs_mkfs_build_tree.part () #6 0x0000000000403c9d in erofs_mkfs_build_tree.part () #7 0x0000000000403c9d in erofs_mkfs_build_tree.part () #8 0x0000000000403c9d in erofs_mkfs_build_tree.part () #9 0x0000000000403c9d in erofs_mkfs_build_tree.part () #10 0x0000000000403c9d in erofs_mkfs_build_tree.part () #11 0x00000000004015db in main () (gdb)
It would be possible to make some small images and benchmark those against Squashfs - but that's no fun. We will likely try again at a later date.
Impressive on paper
Gao Xiang held a presentation on EROFS at the China Open Source Summit 2019.
The PDF with slides titled "EROFS file system introduction" which was used in this presentation has diagrams and tables and charts and they are all very impressive. EROFS appears to use in-place IO decompression which minimizes memory use by using the original read buffer to store decompressed data.
Why it's cool
EROFS is made with mobile phone operating systems like Harmony OS and Android in mind. However, it is also highly interesting for desktop users since all the Linux distributions are using Squashfs for their live USB install images. EROFS has the potential to be a faster and more efficient alternative to Squashfs which means faster installations and more responsive live USB environments.
published 2019-07-22 - last edited 2020-09-22