Waifu2x
waifu2x is a widely praised image up-scaler focused on up-scaling 2D anime images. The original version is Windows-only and it requires CUDA which means you need an NVidia GPU. GNU/Linux users can choose between several versions. There's a Vulkan implementation that works and does a really good job at up-scaling and a OpenGL implementation that does not seem to do anything at all.
There's also a Nvidia CUDA version (for those who find proprietary drivers to be acceptable) and a Caffe version. We have not tested those.
The Myriad Of Versions[edit]
The original author of waifu2x wrote it for Microsoft Windows with Nvidias proprietary CUDA API as a requirement. That Japanese author, who goes by the GitHub handle nagadomi, describes the original waifu2x as
"Image Super-Resolution for Anime-style art using Deep Convolutional Neural Networks. And it supports photo."
The original waifu2x code has been forked by a whole lot of people who mostly produced other Windows-only versions which were later abandoned. It is therefore hard to find any actually useful code or version which runs on GNU/Linux.
The waifu2x-ncnn-vulkan implementation described below runs on GNU/Linux and it works great, the waifu2x-glsl implementation runs on Linux but it's either broken or a bad implementation.
waifu2x-ncnn-vulkan[edit]
| Original author(s) | 是 nihui |
|---|---|
| Stable release | version 20200818
/ August 18, 2020 |
| Repository | github.com /nihui/waifu2x-ncnn-vulkan |
| Bugtracker | github.com /nihui/waifu2x-ncnn-vulkan/issues |
| Written in | C++ |
| Engine | ncnn (network inference computing framework) |
| Type | Image up-scaling |
| License | MIT Software License |
waifu2x-ncnn-vulkan is your best Waifu2x option on GNU/Linux as you will see if you examine the various images below.
A pre-compiled Linux binary is available on the GitHub releases page:
wget https://github.com/nihui/waifu2x-ncnn-vulkan/releases/download/20200818/waifu2x-ncnn-vulkan-20200818-linux.zip
unzip waifu2x-ncnn-vulkan-20200818-linux.zip
cd waifu2x-ncnn-vulkan-20200818-linux
mkdir -p $HOME/bin/waifu2x-ncnn
# Copy models
cp -r models-cunet models-upconv_7_anime_style_art_rgb models-upconv_7_photo $HOME/bin/waifu2x-ncnn
# Copy binary
cp waifu2x-ncnn-vulkan $HOME/bin
You can, alternatively, compile it from source:
mkdir $HOME/src; cd $HOME/src
git clone https://github.com/nihui/waifu2x-ncnn-vulkan.git
cd waifu2x-ncnn-vulkan
# Acquire the sub-modules
git submodule init
git submodule update
# Create and enter build directory
mkdir src/build
cd src/build
cmake ..
# -j$(nproc) makes it use all the CPU cores
make -j$(nproc)
That should create a waifu2x-ncnn-vulkan executable. You'll need that and the models/ folder in the git source tree root (a model needs to be specified with -m when you use it). You can install the fine binary you compiled from source and the models this way:
# Assuming you're still in src/build,
# waifu2x-ncnn-vulkan/ is where you want to go
cd ../..
# Make a folder for it
mkdir -p $HOME/bin/waifu2x-ncnn
# Copy models
cp -r models/* $HOME/bin/waifu2x-ncnn
# Copy binary
cp src/build/waifu2x-ncnn-vulkan $HOME/bin
The options you can/should use are:
-i input-path |
input image path (jpg/png/webp) or directory |
-o output-path |
output image path (jpg/png/webp) or directory |
-n noise-level |
denoise level (-1/0/1/2/3, default=0) |
-s scale |
upscale ratio (1/2, default=2) |
-t tile-size |
tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu |
-m model-path |
waifu2x model path (default=models-cunet) |
-g gpu-id |
gpu device to use (default=auto) can be 0,1,2 for multi-gpu |
-j load:proc:save |
thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu |
-x |
enable tta mode |
-f format |
output image format (jpg/png/webp, default=ext/png) |
You have to specify a "noise model" to use. Making an alias to the "default" (which otherwise requires it to be in the folder you are running it from) makes it easier:
alias waifu2x-ncnn='$HOME/bin/waifu2x-ncnn-vulkan -m $HOME/bin/waifu2x-ncnn/models-cunet'
waifu2x-ncnn-vulkan produces decent results with no options beyond the path to the default model. It does come with two other models: models-upconv_7_anime_style_art_rgb and models-upconv_7_photo. They may produce slightly better results for anime and photos.
waifu2x-glsl[edit]
| Original author(s) | Yuichiro Nakada |
|---|---|
| Stable release | 2018-07-10
/ July 10, 2018 |
| Repository | https://github.com/yui0/waifu2x-glsl |
The waifu2x fork waifu2x-glsl compiles and runs on GNU/Linux (and Windows and macOS). It claims to use the cross-platform glfw library to do OpenGL assisted up-scaling using the waifu2x algorithm.
waifu2x-glsl runs and claims to do something using the GPU (it probably does something). It does not appear to be actually using the waifu2x algorithm or doing anything other than plain up-scaling. The resulting images are .. on par with, or perhaps worse, than up-scaling with ImageMagick's convert or the GNU Image Manipulation Program. You can compile it and try it using these instructions, but it would probably be a complete waste of time.
The latest release is from 2018.
waifu2x-glsl does not appear to do a very good job at up-scaling. Perhaps it doesn't actually use the waifu2x algorithm.
The dependencies you need to compile it on Fedora are:
sudo dnf install glfw-devel mesa-libgbm-devel libdrm-devel mesa-libGL-devel mesa-libGLU-devel mesa-libEGL-devel mesa-libGLES-devel
Get the equivalent build dependencies on for your favorite distribution if it's something else like Debian.
git clone https://github.com/yui0/waifu2x-glsl.git
cd waifu2x-glsl
make
The waifu2x_glsl binary is placed in the root of the source tree. There is no Makefile so you can't run make install. You have to install it yourself.
mkdir -p $HOME/bin/waifu2x-glsl
cp noise1_model.json noise2_model.json $HOME/bin/waifu2x-glsl
cp waifu2x_glsl $HOME/bin
You have to use one of the noise model files with it. waifu2x-glsl appears to do a better job with the scale2.0x_model.json from WL-Amigo/waifu2x-converter-cpp than it does with the included noise models.
waifu2x-glsl will by default look for a noise1_model.json file in the current folder where it is executed. Other files in other locations can be specified by adding a -m path/file.json parameter when you run it.
The desired up-scaling size is specified with -s (2 for 2x, 4 for 4x etc) and an output file needs to be specified with -o.
Note that you can not just place waifu2x_glsl and noise1_model.json in $HOME/bin and expect that to work. It looks for a model in the current working folder, not the folder where the binary lives. You can make put both files in $HOME/bin and make an alias:
alias waifu2x-glsl="$HOME/bin/waifu2x_glsl -m $HOME/bin/waifu2x-glsl/noise1_model.json"
That alias makes it possible to up-scale an image by 2x using
waifu2x-glsl -s 2 inputfile.jpg -o outputfile.jpg
Images[edit]
128x128px "agt_family" icon[edit]
Everaldo Coelho (www.everaldo.com) is a graphics designer who created several free themes such as "Crystal" and "Kids" in the early 2000s. The Kids theme, released in 2003, did not include any SVG images - only icons up to 128x128. "agt_family.png" is one such icon.
Waifu2x has trouble with especially small images with a lot of feathered PNG alpha transparency. Adding a white background with GIMP or other software will usually produce better results, as was done here.
RealSR produced weird artifacts regardless of whether I flattened the image or not. This shows where waifu2x's emphasis on 2D anime images during the training of the neural network comes in.
Original:
ImageMagick convert: convert agt_family.png -resize 256 agt_family.256px.ImageMagick.png
waifu2x_glsl: waifu2x-glsl -s 2 agt_family.png -o agt_family.waifu2x_glsl-2x.png :
waifu2x-ncnn: waifu2x-ncnn -s 2 -i agt_family.png -o agt_family.waifu2x-ncnn-2x.png:
waifu2x-ncnn
|
ImageMagick's "convert"
|
RealSR:
|
RealSR from 64px:
|
waifu2x_glsl
|
gimp
|
Note: RealSR can only do x4 so the RealSR image on the left is down-scaled by 50%.
waifu2x_glsl lacks PNG transparency support. It also appears to be non-functional or not doing a very good job.
the waifu2x-ncnn-vulkan image looks distinctively better than the others. RealSR is also better than GIMP/Convert.
As we begin showing real life photographs it's important to remember that Waifu2x is ALMOST ENTIRELY trained on 2D anime images; the fact that it actually goes nearly toe to toe with RealSR on photographs is impressive.
For non-photographs waifu2x VASTLY outperforms closed-source proprietary algorithms from the likes of Gigapixel and LetsEnhance, even when set to "man-made" on the former and when using the new "digital-art" beta of the latter; both of which produce weird chromatic artifacts and BIZARRE texture artifacts which waifu2x doesn't, and they produce even more than RealSR does.
Here we will include examples of Gigapixel running on wine, which shows how free alternatives compete with a $99 USD piece of closed-source software. LetsEnhance however under performs every other algorithm, and isn't even worth including.
256px WJSN Yeonjung photo[edit]
Original (Actual original was a high-resolution photo that was down-scaled for testing purposes):
GIMP:
|
ImageMagick convert:
|
waifu2x-ncnn-vulkan 2x:
|
RealSR:
|
waifu2x_glsl 2x:
|
Gigapixel AI:
|
Kind of hard to tell what's better here, waifu2x-ncnn-vulkan or RealSR. It's easy to tell that both are better than GIMP/ImageMagick. Gigapixel is MUCH sharper, sure, but it also introduces a lot more aliaisng. Gigapixel AI shines when images are already so high a resolution that you don't notice the extra aliaisng. Also you might not even want your image THAT sharp, and Gigapixel AI doesn't really let you tone down the sharpness even though it gives you a lot of options, you just have to like sharpness.
Satiana[edit]
1920x1080 screenshot of Satiana from Gabriel Dropout.
We down-scaled a screenshot of Satiana from Gabriel Dropout down to just 200x113 pixels using GIMP:
GIMP:
|
ImageMagick convert:
|
RealSR 4x, downscaled for comparison:
|
waifu2x-ncnn-vulkan 2x:
|
The waifu2x-ncnn-vulkan and RealSR images look much better than the GIMP and ImageMagick results. Note that the RealSR realsr-ncnn-vulkan implementation will only do 4x, so the image above is down-scaled from 800px to 400px. You can click on it to see the original RealSR result. I didn't even run this through Gigapixel AI because all it'd do is make it look like a weirdly textured nightmare.
Verdict And Conclusion[edit]
The visible difference between up-scaling the above images in GNU Image Manipulation Program, ImageMagick's convert, waifu2x-glsl, Gigapixel AI, and waifu2x-ncnn-vulkan is a clear case of one isn't like the other: waifu2x-ncnn-vulkan will never introduce color artifacts or weird texturing to images. It might not be the sharpest, but it also never produces a BAD result. Every other one is kind of a gamble to some degree.
We leave you with the following 500x500 image and four only numbered 2x upscaled images of Jennie from the k-pop group Blackpink for your comparison pleasure and/or consideration:
#1:
#2:
#3:
#4:
#5:
#6:
Image 1 was made using GIMPs ▸ , 2 was made using waifu2x-ncnn-vulkan, 3 was made with RealSR, 4 was with Gigapixel AI, 5 was made with waifu2x_glsl and 6 was made using convert -resize.
RealSR seems a lot better suited for some photographs than waifu2x (mostly already-high-resolution ones, and ones with grain), but waifu2x is better at 2D images across the board, and possibly also 3D CGI. Waifu2x seems to be the only algorithm immune to chroma artifacts and the tendency to introduce weird textures to 2D images. Gigapixel is so similar here to RealSR it's hard to justify anybody spending $99 USD on it. Gigapixel AI has a new sister program called Vidoe Enhance AI which actually does still images much better than Gigapixel AI does, but unlike Gigapixel AI it doesn't seem to run under wine.
Alternatives[edit]
RealSR, "Real-World Single Image Super-Resolution", is a image up-scaling program designed for photos. It's really good. It is equal to or better than waifu2x for anime and it is better at up-scaling photos. RealSR will only do 4x up-scaling while waifu2x-ncnn-vulkan will only do 2x.
Enable comment auto-refresher
Gigith
Permalink |
WaiLin
Gigith
Gigith
Permalink |
Anonymous (92752c81eb)
Permalink |