Bash – How to background a shell script during a Kickstart

bashkickstartlinuxshell

I have a Red Hat Kickstart process which reports its progress at key points via a POST request to a status server.

This is fine during %pre and %post, but when the actual build is taking place between them, it's an informational black hole.

I've written a simple shell snippet that reports on the number of packages installed to give a rough idea of progress. I've placed the following in %pre:

%pre

## various other stuff here, all works fine ##

cat > /tmp/rpm_watcher.sh << EOF_RPM
PREV=-1
while true
do
    COUNT="\$(rpm -qa | wc -l)"
    if [ \${COUNT} -ne \${PREV} ] ; then
        /bin/wget --post-data " ${Hostname} : Package count \${COUNT}" ${builddest}/log
        PREV=\${COUNT}
    fi
    sleep 15
done
EOF_RPM
/bin/sh /tmp/rpm_watcher.sh &
disown -a
%end

However, when I launch this as a background task from %pre as above, it hangs waiting for the script to end — %pre never completes (if I kill the spawned script %pre completes and the build proper starts).

I can't use nohup as it isn't available in the pre-installation environment, the same goes for using at now and screen.

I've attempted to use disown -a, which is available; this seems to successfully disown the process (such that it's owned by PID 1) but still it hangs waiting for the script to finish.

Can anyone offer me an alternative?

Best Answer

You were very close to the solution. Anaconda (the installer) is written in Python, so I went digging into the code.

Ultimately, it executes the script like this:

    rc = iutil.execWithRedirect(self.interp, ["/tmp/%s" % os.path.basename(path)],
                                stdin = messages, stdout = messages, stderr = messages,
                                root = scriptRoot)

A bit more digging and you can find iutil.execWithRedirect defined in 'iutil.py'. This function ultimately uses subprocess.Popen (a Python built in) to execute the command. It also tries very hard to get the contents of STDOUT and STDERR from the %pre script.

The code looks like this:

    #prepare tee proceses
    proc_std = tee(pstdout, stdout, program_log.info, command)
    proc_err = tee(perrout, stderr, program_log.error, command)

    #start monitoring the outputs
    proc_std.start()
    proc_err.start()

    proc = subprocess.Popen([command] + argv, stdin=stdin,
                            stdout=pstdin,
                            stderr=perrin,
                            preexec_fn=chroot, cwd=root,
                            env=env)

    proc.wait()
    ret = proc.returncode

    #close the input ends of pipes so we get EOF in the tee processes
    os.close(pstdin)
    os.close(perrin)

    #wait for the output to be written and destroy them
    proc_std.join()
    del proc_std

    proc_err.join()
    del proc_err

So, with what you have, you get past the proc.wait() and os.close calls by forking into the background.

proc_std and proc_err are threads that repeatedly call readline on STDOUT and STDERR. They keep reading until EOF is encountered. Since your script inherits the STDOUT and STDERR socket from the %pre script, they will never encounter EOF. Ananconda then hangs waiting for the thread reading STDOUT to exit (on the 'proc_std.join()') line, which never happens.

It's a very confusing issue, but ultimately a very simple fix. Instead of:

/bin/sh /tmp/rpm_watcher.sh &

use

/bin/sh /tmp/rpm_watcher.sh > /dev/null 2>&1 < /dev/null &

This ensures your script doesn't inherit STDOUT and STDERR, so Anaconda doesn't hang and the install can continue.

Related Question