Centos – Setting ulimit correctly for PHP on Linux

centoslinuxSecurityulimit

I have /etc/security/limits.conf set to the following:

nobody soft nofile 409600
nobody hard nofile 1024000
phpuser soft nofile 409600
phpuser hard nofile 1024000
httpd soft nofile 409600
httpd hard nofile 1024000
* soft nofile 409600
* hard nofile 1024000

However, php pages still display:

[...] failed to open stream: Too many open files [...]

Setting ulimit -n 10000 looks to only be a temporary fix.

I also have the following set:

fs.file-max = 20970800
net.core.somaxconn = 1024000
kern.maxfilesperproc = 16638400
kern.maxfiles = 819200

Best Answer

I see two potential problems.

Your limit may not apply to phpuser

phpuser may ignore your new limit because it might not use PAM to "log-in", so /etc/security/limits.conf would not apply. See this answer for more details.

The system wide limit is reached

Your are changing users processes limits. The kernel also has a system wide limit on the number of open file-handles.

This might be your problem. You can check the value like this:

$ sysctl fs.file-max
$ sysctl fs.file-nr

By default, file-max shoud be 10% of your available system memory in kB, which may not be a very big number: 4 GB RAM ~ 400000 files which is below the limit you are trying to set.

The documentation says:

The three values in file-nr denote the number of allocated file handles, the number of allocated but unused file handles, and the maximum number of file handles. Linux 2.6 always reports 0 as the number of free file handles -- this is not an error, it just means that the number of allocated file handles exactly matches the number of used file handles.

Related Solutions

Ubuntu – Unable to change ulimit setting

Check the manual: man limits.conf

That will show you that the first field in that configuration file is the <domain> or simplified: the users or groups for whom the setting is applied to. So when you use root there it's completely expected that it only works for root and not for any other user.

Try the following to apply it to all users or better find some middle ground and apply to the correct users/groups.

*     hard      nofile     20000

Linux – Is setting ulimit -v sufficient to avoid memory leak

Some description about how ulimit works:

ulimit has deal with setrlimit and getrlimit system calls. It's easy to ensure by strace-ing of bash process (ulimit is component of the bash). I set 1024kb of max memory size:

$ ulimit -m 1024

In another console:

$ strace -p <my_bash_pid>
. . .
getrlimit(RLIMIT_RSS, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
setrlimit(RLIMIT_RSS, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
. . .

setrlimit man page write next about RLIMIT_RSS:

RLIMIT_RSS Specifies the limit (in pages) of the process's resident set (the number of virtual pages resident in RAM). This limit only has effect in Linux 2.4.x, x < 30, and there only affects calls to madvise(2) specifying MADV_WILLNEED.

madvice syscall is just advice to kernel and kernel may ignore this advice. Even bash man page about ulimit write following:

-m The maximum resident set size (many systems do not honor this limit)

That is the reason why -m doesn't work.

About -v option:

I set 1024 kb of virtual memory:

$ ulimit -v 1024

In another console:

$ strace -p <my_bash_pid>
. . .
getrlimit(RLIMIT_AS, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
setrlimit(RLIMIT_AS, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
. . .

setrlimit man page write next about RLIMIT_AS:

RLIMIT_AS The maximum size of the process's virtual memory (address space) in bytes. This limit affects calls to brk(2), mmap(2) and mremap(2), which fail with the error ENOMEM upon exceeding this limit. Also automatic stack expansion will fail (and generate a SIGSEGV that kills the process if no alternate stack has been made available via sigaltstack(2)). Since the value is a long, on machines with a 32-bit long either this limit is at most 2 GiB, or this resource is unlimited.

Program consist of 3 segments (data, code, stack) compose virtual program memory space.

Code segment is const and contain program instructions.
Data segment is controlled by following:

brk syscall adjusts size of data segment (part of virtual memory) of the program.

mmap syscall maps file or device to virtual memory of process.

Many programs allocates memory (direct or indirect) by calling standard function from C Library (malloc) which allocates memory from heap (part of data segment). malloc adjust size of data segment by calling brk syscall.
Stack stores functions variables (variable takes memory during allocation from stack).

So, that's why the -v option is works for you.

If -v is sufficient for your task, then there is no reasons to do something else and it's sufficient.

If you want to take control about huge count of specific memory features for process (memory pressure, swap usage, RSS limit, OOM and so on) I suggest to you to use cgroups memory capabilities.

If your application is a service I suggest to you to use systemd slice features, as the most convenient for controlling and limiting resources of service or group of services (also it easy to configure instead of configuring cgroups directly) which is managed by systemd.