HOWTO deal with "Too many open files" errors by increasing the limit

Linux has a limit for how many files, real files or other, a single piece of software can keep open. There is also per-user limits. These are in principle security measures which is why the file /etc/security/limits.conf needs to be changed in order to increase the limits.

Increasing your limit

You should increase your "limits" if you are trying to do something under your username and you get "Too many open files". There's two limits, a "soft" limit and a hard limit. You will need to adjust both. The first thing to do is to run the commands ulimit -a which will list all your soft limits and ulimit -a -H will will list the hard limits.

The output will include something like:

max memory size    (kbytes, -m) unlimited
open files                 (-n) 1024
pipe size       (512 bytes, -p) 8

A maximum of 1024 open files may seem like a lot but in reality it really isn't. Now that you know your limit you may want to adjust it. If your username is, as an example, mysupername and you want a limit of 49152 you add this to /etc/security/limits.conf:

mysupername hard nofile 65536
mysupername soft nofile 49152

If a system service or daemon exits or crashes because of "Too many files" you add the user it is running under to limits.conf. If you are running the electrum software under the user-name electrum it would similarly be:

electrum hard nofile 65536
electrum soft nofile 49152

Don't follow bad examples

Many examples on the Internet propose that you solve a "Too many open files" by adding the following silly directives to your limits configuration file:

*         hard    nofile      500000
*         soft    nofile      500000
root      hard    nofile      500000
root      soft    nofile      500000

Will that work? Yes. It will increase everyone's limit You, your sisters, your dbus manager, your audio daemon, everyone and everything gets to open 500000 files. There is a reason Linux defaults to a fairly low amount of open files per process. You should not blindly increase everyone and everything's limit when you run into a situation where just one process running under one particular user needs a higher limit.

Increasing the total system limit

Linux has one and you can view it by looking at /proc/sys/fs/file-max. It is likely at a high default of something like 1572748 open files.

You can change this by setting fs.file-max = and a value in a file in /etc/sysctl.d/ which ends with .conf. Example:

echo 'fs.file-max = 4718244 > /etc/sysctl.d/88-file-max.conf'

Run sysctl -p to make new SysCtl settings take effect.

Do be aware that there is likely something else going wrong if you believe you need to change fs.file-max. The default is much higher than any limit you would realistically want to give a single user process.

Investigating why is that process is opening a lot of files

You can list all file handles being held open by all instances of a process named process-name by running this as root:

for pid in `ps -C process-name -o pid=`; do ls -l "/proc/$pid/fd"; done

As a practical example, listing all the files held open by java processes could be done with:

for pid in `ps -C java -o pid=`; do ls -l "/proc/$pid/fd"; done

This will not tell you why the process you investigate is opening all the files but it may be helpful to know which. This will, of course, be a rather long list if you are exceeding the open files limit.