Squashfs-tools-ng 1.1.0 Brings Broken tar2sqfs Compressed Tarball Support

From LinuxReviews
Jump to navigationJump to search

squashfs-tools-ng, a independent implementation of squashfs-tools, has made a new release with support for creating squashfs images using shell globing filters and support for creating squashfs images directly from tarball archives (ie .tar.bz2 and tar.xz) using a tool called tar2sqfs. It doesn't work as advertised, but it is there and the not-functional support for archives is new.

written by 윤채경 (Yoon Chae-kyung)  2021-03-29 - last edited 2021-03-30. © CC BY

Tar2sqfs from squashfs-tools-ng-1.1.0.jpg
The tar2sqfs in squashfs-tools-ng should, according to the release-notes, be able to handle tar archives. It doesn't.

squashfs is a compressed image file system somewhat similar to the ISO images you may be familiar with if you ever downloaded and installed a GNU/Linux distribution.

squashfs-tools-ng is one of two competing user-space implementations of command-line utilities for creating and managing squashfs images. Support for mounting and using squashfs images on Linux is in the kernels CONFIG_SQUASHFS driver.

squashfs-tools-ng was released on March 28th, 2021.

Notice the -ng in squashfs-tools-ng

squashfs-tools-ng is not squashfs-tools, they are completely different user-space utilities for squashfs. squashfs-tools development seemingly sort of died off a few years ago. A few developers decided to fork it and actively develop that fork. Those developers looked at the code they had forked, scratched their heads and decided to just scrap the whole thing and start over from scratch. Their efforts, which can be seen at their repository at github.com/AgentD/squashfs-tools-ng, is what is now known as squashfs-tools-ng.

The original squashfs-tools did not die; it just appear that way to the squashfs-tools-ng developers. The repository for the original utility is now maintained at github.com/plougher/squashfs-tools.

The latest squashfs-tools-ng version adds support for turning compressed tar archives (.tar.bz2 and tar.xz files) into squashfs with a tool called tar2sqfs. The competing squashfs-tools (notice no -ng) utilities has no such tool.

The newfound support for piping compressed archives into tar2sqfs does not appear to be all that .. functional.

tar2sqfs  -c zstd linux-5.11.10.squashfs < linux-5.11.10.tar.xz

produces an error about unexpected end-of-file even though tar xf linux-5.11.10.tar.xz unpacks the Linux archive from kernel.org just fine.

tar2sqfs works not really better if a .tar.bz2 archive is piped into it.

tar2sqfs  -c zstd linux-5.11.10.squashfs < linux-5.11.10.tar.bz2

produces a squashfs image with 27 seemingly random files from the linux-5.11.10.tar.bz2 archive. That's slightly less than the 71239 files in it, but it's something.

tar2sqfs in squashfs-tools-ng 1.1.0 is, overall, not very impressive (or functional).

There's also a new "glob" keyword in the gensquashfs utility in squashfs-tools-ng 1.1.0. It works similar to the find command; you can use it to create squashfs images based on file types and shell globing filters in a directory structure.

The source code for squashfs-tools-ng 1.1.0 for Linux can be acquired from infraroot.at/pub/squashfs/. It is a quick and strait-forward compile since there basically no required dependencies:

wget https://infraroot.at/pub/squashfs/squashfs-tools-ng-1.1.0.tar.xz
tar xf squashfs-tools-ng-1.1.0.tar.xz
cd squashfs-tools-ng-1.1.0/
make -j$(nproc)
sudo make install

There is also a Windows version, made using the MinGW toolchain, available at infraroot.at/pub/squashfs/windows/.

(0 votes)


Anonymous user #1

13 days ago
Score 0++
Just a thought: What if you unpack the archive and pipe the unpacked tar into the tool?


13 days ago
Score 1++

The above is incorrect, squashfs-tools development never died, I simply moved the main development repository from Sourceforge to Github. The Sourceforge repository has been developmentally dormant for many years and is largely out of date. But it can't be deleted because a lot of installations/buildroot-kits still point-to-it for older versions. Deletion will break these systems.

Some people forked Squashfs-tools from the Sourceforge repository, to add "reproducible builds" functionality to Mksquashfs. They did this badly, and broke the behaviour of Mksquashfs. Afterwards they blamed the Squashfs-tools source code for their failure to get the patch working, and the many bug reports received, claiming it was in an "unmaintainable state", which was a complete falsehood. Having failed to alter Mksquashfs, they started to write their own. This is not because Squashfs-tools was unmaintained, but, simply because they couldn't get it to work for them.

As Squashfs author and maintainer I first became aware of this "reproducible builds" patch when bug reports began to crop up on my Squashfs-tools repository on GitHub. For instance https://gith...ls/issues/17.

There I gave an analysis of why the patch broke Mksquashfs (https://gith...nt-278862875).

I correctly added support for "reproducible builds" in my 4.4 release of Squashfs-tools in August 2019 (see https://gith...ls/issues/54 and https://gith...r/README-4.4).

Please do not repeat the falsehood that Squashfs-tools were unmaintained.
Add your comment
LinuxReviews welcomes all comments. If you do not want to be anonymous, register or log in. It is free.