The setrlimit(2) syscall is relevant to limit resources (CPU time -integral number of seconds, so at least 1 sec- with RLIMIT_CPU
, file size with RLIMIT_FSIZE
, address space with RLIMIT_AS
, etc...). You could also set up disk quotas. The wait4(2) syscall tells you -and gives feedback- about some resource usage. And proc(5) tells you a lot more, and also getrusage(2) (you might code some monitor which periodically stops the entire process group using SIGSTOP
, call getrusage
or query /proc/$PID/
, then send SIGCONT
-to continue- or SIGTERM
-to terminate- to that process group).
The valgrind tool is very useful on Linux to help finding memory leaks. And strace(1) should be helpful too.
If you can recompile the faulty software, you could consider -fsanitize=address
and -fsanitize=undefined
and other -fsanitize=
... options to a recent version of the GCC compiler.
Perhaps you have some batch processing. Look for batch monitors, or simply code your own thing in C, Python, Ocaml, Perl, .... (which forks the command, and loop on monitoring it...). Maybe you want some process accounting (see acct(5) & sa(8)...)
Notice that "amount of memory used" (a program generally allocates with mmap
& releases memory with munmap
to the kernel while running) and "CPU time" (see time(7), think of multi-threaded programs ...) are very fuzzy concepts.
See also PAM and configure things under /etc/security/
; perhaps inotify(7) might also be helpful (but probably not).
Read also Advanced Linux Programming and syscalls(2)
I figured out how to do what I was describing, but it was a bit counter-intuitive, so I'm posting the answer here for people who might hit this page when searching (tl:dr; at bottom). As far as I know, there is no blanket way to just filter out processes with a certain PID from ftrace as easily as it is to tell it to ONLY consider processes with a certain PID, but in my case, I only care about raw system calls (sys_enter) and I found out how to eliminate records with certain PIDs from being included for those and this is how:
The ftrace directory is:
/sys/kernel/debug/tracing/
Inside, there is a directory called "events." From here, you can see all the things that ftrace can trace, but for my case, I go into "raw_syscalls."
Within raw_syscalls," the two subdirectories are sys_enter and sys_exit.
Within sys_enter (and sys_exit, for that matter), there are the following files:
enable
filter
format
id
trigger
"filter" is the one we care most about right now, but format has useful information regarding the fields of an entry produced by ftrace when sys_enter is enabled:
name: sys_enter
ID: 17
format:
field:unsigned short common_type; offset:0; size:2; signed:0;
field:unsigned char common_flags; offset:2; size:1; signed:0;
field:unsigned char common_preempt_count; offset:3; size:1; signed:0;
field:int common_pid; offset:4; size:4; signed:1;
field:long id; offset:8; size:8; signed:1;
field:unsigned long args[6]; offset:16; size:48; signed:0;
Here, we care about common_pid.
If you want your trace to omit records from a process with PID n, you would edit
/sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/filter
To read:
common_pid != n
If the program you're trying to ignore while tracing has multiple threads or multiple processes, you just use the && operator. Say you want to omit processes with PIDs n, o, and p, you would edit the file so that it reads:
common_pid != n && common_pid != o && common_pid != p
To clear a filter, you just write "0" to the file:
echo "0" > /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/filter
...would do the trick.
enable has to contain "1" for the event you're tracing as well as tracing_on in the ftrace directory. Writing in 0 turns tracing of that event (or all tracing in the case of tracing_on) off.
Writing to these files requires root permissions.
That's about all I can think of. Thanks to anyone who read/voted on this and I hope my answer helps someone. If anyone knows a way that makes the way I did it look stupid, feel free to call me out.
tl;dr: to filter out records from process 48, write:
common_pid != 48
...to
/sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/filter
Filter multiple PIDs (eg. 48, 49, 53, 58) by writing this instead:
common_pid != 48 && common_pid != 49 && common_pid != 53 && common_pid !=58
Replace "events/raw_syscalls/sys_enter" with your desired event and replace my numbers with whatever PIDs you want to ignore.
Best Answer
Linux-specific answer:
perf-tools contains an execsnoop that does exactly this. It uses various Linux-specific features such as ftrace. On Debian, its in the perf-tools-unstable package.
Example of me running
man cat
in another terminal:I doubt there is a portable way to do this.