Linux – Why is the Number of Open Files Limited?

Most of the values¹ in limits.conf are limits that can be set with the ulimit shell command or the setrlimit system call. They are properties of a process. The limits apply independently for each process. In particular, each process can have up to nofile open files. There is no limit to the number of open files cumulated by the processes of a user.

The nproc limit is a bit of a special case, in that it does sum over all the processes of a user. Nonetheless, it still applies per-process: when a process calls fork to create a new process, the call is denied if the number of processes belonging to the process's euid is would be larger than the process's RLIMIT_NPROC value.

The limits.conf man page explains that the limits apply to a session. This means that all the processes in a session will all have these same limits (unless changed by one of these processes). It doesn't mean that any sum is done over the processes in a session (that's not even something that the operating system tracks — there is a notion of session, but it's finer-grained than that, for example each X11 application tends to end up in its own session). The way it works is that the login process sets itself some limits, and they are inherited by all child processes.

¹ _{The exceptions are maxlogins, maxsyslogins and chroot, which are applied as part of the login process to deny or influence login.}

The RLIMIT_NOFILE is on the maximum file descriptor value you may obtain/allocate, not on how many files may be open at a time.

Child processes inherit the limit, but other than that there's nothing that a child may do to influence the parent here. If the parent has some free fds in the range 0->limit-1, then it will still be able to open new files (wrt to that limit) regardless of what any of its children does (you may run into other global limits though).

In any case, note that if the limit is say 500, you can still have more than 500 file descriptors open if you had some that were open (including in parent processes) prior to the limit being lowered.

$ bash -c 'exec  1023> /dev/null; ulimit -n 500;
   command exec 600> /dev/null; ls -l /proc/self/fd; exit'
bash: 600: Bad file descriptor
total 0
lrwx------ 1 chazelas chazelas 64 Jun 17 08:40 0 -> /dev/pts/1
lrwx------ 1 chazelas chazelas 64 Jun 17 08:40 1 -> /dev/pts/1
l-wx------ 1 chazelas chazelas 64 Jun 17 08:40 1023 -> /dev/null
lrwx------ 1 chazelas chazelas 64 Jun 17 08:40 2 -> /dev/pts/1
l-wx------ 1 chazelas chazelas 64 Jun 17 08:40 3 -> /dev/null
lr-x------ 1 chazelas chazelas 64 Jun 17 08:40 4 -> /proc/8034/fd

That process running ls has a limit of 500 there inherited from its parent (so can't get a new fd bigger than 499). Still it does have the fd 1023 open.