Vulkan Applications Running On Linux With Older Mesa Versions Could Eat ALL Your RAM
The Mesa Vulkan Linux drivers had a very unfortunate bug that would cause Vulkan applications to fill all GPU memory, all system memory and eventually all of swap too if textures are shared between Vulkan and OpenGL using external memory. This is why our test of Chromium 84 with Vulkan enabled resulted in a disaster where all types of memory were filled. Mesa <=20.1.5, the upcoming Mesa 20.2 release and Mesa git have a fix.
written by 林慧 (Wai Lin) 2020-09-06 - last edited 2020-09-07. © CC BY
Chromium 84 with Vulkan enabled and MangoHud showing that it is using 7.5 GiB GPU memory and 27.9 GiB system memory showing two tabs, one of which is blank.
Software developer Christoph Haag reported a rather nasty Mesa bug during the course of his work on the xrdesktop project in September 2019. It turns out that Mesa did not bother with freeing memory used for textures in external memory that are shared between OpenGL and Vulkan.
The major memory leak affected anything that would:
- create vulkan texture
- export fd for vulkan texture memory with external memory
- import fd into opengl texture with memory object
- delete opengl texture
- destroy vulkan texture (vkFreeMemory etc.)
Any textures created and destroyed in this matter would not actually be freed from memory until the application who created the textures was terminated. This bug remained in Mesa for almost a year until it was finally fixed by Swiss Google-employed developer Bas Nieuwenhuizen with commit commit 440275b3 on August 1st, 2020.
GNU/Linux distributions shipping Mesa versions prior to 20.1.5 have this rather nasty OpenGL/Vulkan memory leak. Mesa 20.2, expected to be released next week, will not have this problem.
We noticed the affects of this bug ourselves when we tested the effects of enabling Vulkan rendering in Chromium 84. The result of doing that was a scandal: Just switching between browser tabs resulted in a gradual GPU increase followed by a gradual system memory increase until all GPU VRAM, all system memory and all swap was filled. The Linux Out Of Memory (OOM) killer would eventually kick in and eradicate the Chromium GPU process.
Chromium 85 on Linux 5.9rc3 and Mesa 20.2.0 rc4 does not eat all the GPU VRAM when Vulkan is enabled on newer Mesa versions but it will very gradually eat all system RAM.
Chromium 85+Mesa 20.2.0rc4 eats system RAM, but not GPU VRAM, when Vulkan is enabled.
Chromium's slow consumption of all system RAM, but not GPU VRAM, when Vulkan is enabled does not appear to have anything to do with the Mesa drivers. It may have more to do with Chromium 85+ using the ANGLE graphics layer with a bundled OpenGL ES 2.0 library than it has to do with Mesa.
Mesa 20.1.6 is the latest stable Mesa release so the Linux distributions that do not insist on using ancient Mesa versions will hopefully make a Mesa 20.1.5+ version available to their users in the near future. A bug that makes many Vulkan games and programs eventually fill all GPU and system memory really is a good reason to make an updated version available. Those using LTS distributions and distributions like Endless OS who ship very old Mesa libraries like 19.1.7 will just have to live with this bug and, if possible, avoid any and all Vulkan applications.
Enable comment auto-refresher