Shell – What Happens When Executing a File

execkernelshebangshell

So, I thought I had a good understanding of this, but just ran a test (in response to a conversation where I disagreed with someone) and found that my understanding is flawed…

In as much detail as possible what exactly happens when I execute a file in my shell? What I mean is, if I type: ./somefile some arguments into my shell and press return (and somefile exists in the cwd, and I have read+execute permissions on somefile) then what happens under the hood?

I thought the answer was:

  1. The shell make a syscall to exec, passing the path to somefile
  2. The kernel examines somefile and looks at the magic number of the file to determine if it is a format the processor can handle
  3. If the magic number indicates that the file is in a format the processor can execute, then
    1. a new process is created (with an entry in the process table)
    2. somefile is read/mapped to memory. A stack is created and execution jumps to the entry point of the code of somefile, with ARGV initialized to an array of the parameters (a char**, ["some","arguments"])
  4. If the magic number is a shebang then exec() spawns a new process as above, but the executable used is the interpreter referenced by the shebang (e.g. /bin/bash or /bin/perl) and somefile is passed to STDIN
  5. If the file doesn't have a valid magic number, then an error like "invalid file (bad magic number): Exec format error" occurs

However someone told me that if the file is plain text, then the shell tries to execute the commands (as if I had typed bash somefile). I didn't believe this, but I just tried it, and it was correct. So I clearly have some misconceptions about what actually happens here, and would like to understand the mechanics.

What exactly happens when I execute a file in my shell? (in as much detail is reasonable…)

Best Answer

The definitive answer to "how programs get run" on Linux is the pair of articles on LWN.net titled, surprisingly enough, How programs get run and How programs get run: ELF binaries. The first article addresses scripts briefly. (Strictly speaking the definitive answer is in the source code, but these articles are easier to read and provide links to the source code.)

A little experimentation show that you pretty much got it right, and that the execution of a file containing a simple list of commands, without a shebang, needs to be handled by the shell. The execve(2) manpage contains source code for a test program, execve; we'll use that to see what happens without a shell. First, write a testscript, testscr1, containing

#!/bin/sh

pstree

and another one, testscr2, containing only

pstree

Make them both executable, and verify that they both run from a shell:

chmod u+x testscr[12]
./testscr1 | less
./testscr2 | less

Now try again, using execve (assuming you built it in the current directory):

./execve ./testscr1
./execve ./testscr2

testscr1 still runs, but testscr2 produces

execve: Exec format error

This shows that the shell handles testscr2 differently. It doesn't process the script itself though, it still uses /bin/sh to do that; this can be verified by piping testscr2 to less:

./testscr2 | less -ppstree

On my system, I get

    |-gnome-terminal--+-4*[zsh]
    |                 |-zsh-+-less
    |                 |     `-sh---pstree

As you can see, there's the shell I was using, zsh, which started less, and a second shell, plain sh (dash on my system), to run the script, which ran pstree. In zsh this is handled by zexecve in Src/exec.c: the shell uses execve(2) to try to run the command, and if that fails, it reads the file to see if it has a shebang, processing it accordingly (which the kernel will also have done), and if that fails it tries to run the file with sh, as long as it didn't read any zero byte from the file:

        for (t0 = 0; t0 != ct; t0++)
            if (!execvebuf[t0])
                break;
        if (t0 == ct) {
            argv[-1] = "sh";
            winch_unblock();
            execve("/bin/sh", argv - 1, newenvp);
        }

bash has the same behaviour, implemented in execute_cmd.c with a helpful comment (as pointed out by taliezin):

Execute a simple command that is hopefully defined in a disk file somewhere.

  1. fork ()
  2. connect pipes
  3. look up the command
  4. do redirections
  5. execve ()
  6. If the execve failed, see if the file has executable mode set. If so, and it isn't a directory, then execute its contents as a shell script.

POSIX defines a set of functions, known as the exec(3) functions, which wrap execve(2) and provide this functionality too; see muru's answer for details. On Linux at least these functions are implemented by the C library, not by the kernel.

Related Question