I've seen the message SQL2043N Unable to start a child process or thread
occur every once in a while since upgrading to rhel7 and db2 10.5. db2diag.log does not reveal much out of interest, beside:
FUNCTION: DB2 UDB, oper system services,
sqlo_waitlist::initialize, probe:10
MESSAGE : SysV semget failed with errno:
DATA #1 : Hexdump, 4 bytes
The target machines are used for unit-testing and development and relevant kernel parameters seems appropriate:
]$ cat /proc/meminfo
MemTotal: 12140640 kB
]$ ipcs -l
[...]
------ Semaphore Limits --------
max number of arrays = 3072
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767
At the moment there are no databases on the machine, yet – what I believe – a huge number of semaphores is associated with db2fenc1:
]$ ipcs -s | grep db2fenc1 | wc -l
2800 [1]
[1] The number 2800 is not the exact number, but it was close to this
]$ db2stop force
has no effect on the number of semaphores in ipcs -s
. To release the semaphores I had to do ipclean
The only thing I can think of that might be causing db2fenc1 to grab a lot of semaphores is a fenced java function:
CREATE FUNCTION NYA.REMOVE_DIACRITICS( S VARCHAR(50))
RETURNS varchar(50)
FENCED THREADSAFE
DETERMINISTIC
NO SQL
LANGUAGE JAVA
PARAMETER STYLE JAVA
RETURNS NULL ON NULL INPUT
EXTERNAL NAME
'StringUtil:se.vhs.nya.framework.common.util.db.DiacriticUtil!removeDiacritics'
NO EXTERNAL ACTION @
The functions is used in an generated column:
CREATE TABLE ...
( [...]
, <COLNAME2> VARCHAR(25) NOT NULL
GENERATED ALWAYS AS ( NYA.REMOVE_DIACRITICS( <COLNAME1> ) )
, [...] ) @
but there is no massive insert/updates against this table during testing.
Has anyone experienced something similar where semaphores are not released, and in that case, what was the solution?
PS. Not sure if it is relevant, but the JAVA_HEAP_SZ
is:
(JAVA_HEAP_SZ) = 2048
Best Answer
Before reporting this as error, the max number of processes (ulimit -u) should be checked and increased for the instance owner. I'm sure they would ask it anyway.