RealSR
| Original author(s) | Tencent YouTu Lab
|
|---|---|
| Initial release | v20200530 May 30, 2020 |
| Stable release | 20210210
/ February 10, 2021 |
| Repository | github.com /nihui/realsr-ncnn-vulkan |
| Engine | ncnn (network inference computing framework) |
| Operating system | Linux, Windows, macOS |
| Size | 61 Mib 68 MiB installed |
| Type | Image up-scaling |
| License | MIT Software License |
| Website | github.com /jixiaozhong/RealSR |
Real-World Super-Resolution via Kernel Estimation and Noise Injection is a photo up-scaling program. It won a contest called CVPR NTIRE 2020 Challenge on Real-World Super-Resolution in "both tracks". A Linux version using Vulkan (realsr-ncnn-vulkan) is available. RealSR does a really good job at up-scaling photos. It is better than Waifu2x when up-scaling photos and comparable to it when up-scaling anime and other cartoons.
The researchers who created RealSR have written a 8 page long paper (PDF) explaining how it works. It's got pseudo-code and formulas and all kinds of details. The very short version of it is: It's really good at up-scaling images.
Installation[edit]
wget https://github.com/nihui/realsr-ncnn-vulkan/releases/download/20210210/realsr-ncnn-vulkan-20210210-ubuntu.zip
unzip realsr-ncnn-vulkan-20210210-ubuntu.zip
cd realsr-ncnn-vulkan-20210210-ubuntu/
mkdir -p $HOME/bin/realsr-ncnn-vulkan-models
cp -r models-DF2K models-DF2K_JPEG $HOME/bin/realsr-ncnn-vulkan-models
cp realsr-ncnn-vulkan $HOME/bin
Windows and macOS users can acquire the technology from the GitHub Releases Page.
Usage[edit]
-i input-path |
input image path (jpg/png/webp) or directory |
-o output-path |
output image path (jpg/png/webp) or directory |
-s scale |
upscale ratio (4, default=4) |
-t tile-size |
tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu |
-m model-path |
realsr model path (default=models-DF2K_JPEG) |
-g gpu-id |
gpu device to use (default=auto) can be 0,1,2 for multi-gpu |
-j load:proc:save |
thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu |
-x |
enable tta mode |
-f format |
output image format (jpg/png/webp, default=ext/png) |
Using -m to a full path where the model data is stored is required. Creating an alias for that is helpful if you followed the installation instructions above:
alias realsr='$HOME/bin/realsr-ncnn-vulkan -m $HOME/bin/realsr-ncnn-vulkan-models/models-DF2K_JPEG'
That alias lets you simply type realsr -i inputfile.jpg -o outputfile.jpg when you want to convert a file.
Note: The models-DF2K appears to produce bad images when JPEG is up-scaled. models-DF2K_JPEG produces fine images when the original is a JPEG.
Also note: realsr-ncnn-vulkan doesn't actually have any -s options in the current implementation! The default, 4, is your only option. 2, 8, everything else results in invalid scale argument.
Examples[edit]
Satiana[edit]
1920x1080 screenshot of Satiana from Gabriel Dropout.
We down-scaled a screenshot of Satiana from Gabriel Dropout down to just 200x113 pixels using GIMP:
Up-scaled using ImageMagick convert:
Up-scaled using realsr-ncnn-vulkan:
RealSR does a rather impressive job when it's only got a small 200x113 pixel image to work with. That is specially true when you compare it to ImageMagick's convert.
waifu2x-ncnn-vulkan 2x, for comparison:
realsr-ncnn-vulkan will only let you do 4x up-scaling and waifu2x-ncnn-vulkan will only let you do 2x up-scaling which makes it hard to do a 1:1 comparison. This is why the waifu2x image above is smaller that the images created by realsr-ncnn-vulkan and ImageMagick's convert.
AMD Representative[edit]
400 × 548 pixels.
One finer detail stands out as worth considering when it comes to this image:
| file | size | resolution |
|---|---|---|
| AMD_GIRL.jpg (original) | 112K | 400x548 |
| AMD_GIRL_convert_4x.jpg | 732K | 1600x2192 |
| AMD_GIRL_RealSR_4x.jpg | 4.3M | 1600x2192 |
Up-scaled using ImageMagick convert (click to enlarge):
Up-scaled 4x using RealSR:
Upscaled 6x using Gigapixel AI, downscaled with GIMP's Nohalo to match.
I recommend opening different images in different tabs, zooming in, and switching between to really tell the difference.
The RealSR image looks really good except for her arm against the green background, that specific part of the image just doesn't look right. Those kinds of artifacts almost never happen with something like waifu2x, but waifu2x would make it look a lot more cartoony overall.
While Gigapixel AI mostly avoids that chromatic artifact, it makes the text at the bottom almost illegible, makes the clipboard ring-binded, and adds what looks like lumps to her necklace. It also assumes way too much about what was actually on her nametag.
RealSR makes her face and hair smoother and higher resolution looking, but at the cost of looking more un-real. Gigapixel really shines when it comes to faces and hair, it seems, although it brings a sort of "jpeg artifact chroma feel" to the new hair and face.
It's interesting to see what happens when the RealSR image is down-scaled and placed side by side with the original: It looks sharper and overall better down-scaled down to the original image's resolution.
| Original:
|
RealSR:
|
The down-scaled RealSR (using ImageMagick) looks a bit better than the original image.
The Akerman[edit]
Original, 595 × 429:
The Akerman appeared in the background of a picture meant to capture something else. There is no higher resolution, this a crop-out of the Akerman from a larger image and 595 × 429 is the area it filled in the original image.
ImageMagick convert:
RealRS:
The Akerman looks better, but not stunning, after being up-scaled with Akerman. There's not much anyone can do to make that thing look stunning, so it's understandable that the improvement does not make a huge difference.
Verdict And Conclusion[edit]
RealSR is a really amazing image up-scaling program.
It's simply the, by far, best piece of software for that purpose.
Alternatives[edit]
The Waifu2x waifu2x-ncnn-vulkan implementation is almost as good as RealSR or, in some cases, equal to it up-scaling cartoons and drawings. Waifu2x was designed for cartoons, while RealSR is made for real-world pictures.
Footnotes[edit]
Enable comment auto-refresher
Gigith
Permalink |