I had a hard time to find some more information about short lived processes when I was building Foreman SELinux policy for OpenStack Installer (Staypuft). I wanted to see process name and all the parameters for all possible processes spawned on my system. My plan was to write a SystemTap script, but thank to Mirek Grepl and Steve Grubb I learned much easier trick to do that. This assumes you have auditd daemon installed (RHEL/Fedoras are fine):
auditctl -a exit,always -S execve
Then tail your /var/log/audit/audit.log to see messages. Warning: This can flood your system heavily. You will see some decent content there like:
type=CWD msg=audit(1401202071.000:14728): cwd="/" type=PATH msg=audit(1401202071.000:14728): item=0 name="/usr/sbin/ps" nametype=UNKNOWN type=SYSCALL msg=audit(1401202071.000:14729): arch=c000003e syscall=59 success=yes exit=0 a0=7fe10bbe6369 a1=7fe10bbe7580 a2=7fff0b1d3208 a3=0 items=2 ppid=26400 pid=5161 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=875 comm="ps" exe="/bin/ps" subj=unconfined_u:system_r:passenger_t:s0 key=(null) type=EXECVE msg=audit(1401202071.000:14729): argc=5 a0="ps" a1="-o" a2="pid,ppid,%cpu,rss,vsize,pgid,command" a3="-p" a4="26974,26817"
I was able to tell that mod_passenger is spawning /bin/ps every few seconds.
This is very useful when tracking down some processes with long puppet runs.
To turn the audit rule down, replace
-a option with
auditctl -d exit,always -S execve
You can filter out audit logs, see auditctl manual page. For example I was interested in processes running in passenger_t SELinux domain:
auditctl -a exit,always -S execve -F subj_type=passenger_t
To add rules permanently, edit
/etc/audit/audit.rules file and restart