So, I thought I had a good understanding of this, but just ran a test (in response to a conversation where I disagreed with someone) and found that my understanding is flawed…
In as much detail as possible what exactly happens when I execute a file in my shell? What I mean is, if I type: ./somefile some arguments
into my shell and press return (and somefile
exists in the cwd, and I have read+execute permissions on somefile
) then what happens under the hood?
I thought the answer was:
- The shell make a syscall to
exec
, passing the path tosomefile
- The kernel examines
somefile
and looks at the magic number of the file to determine if it is a format the processor can handle - If the magic number indicates that the file is in a format the processor can execute, then
- a new process is created (with an entry in the process table)
somefile
is read/mapped to memory. A stack is created and execution jumps to the entry point of the code ofsomefile
, withARGV
initialized to an array of the parameters (achar**
,["some","arguments"]
)
- If the magic number is a shebang then
exec()
spawns a new process as above, but the executable used is the interpreter referenced by the shebang (e.g./bin/bash
or/bin/perl
) andsomefile
is passed toSTDIN
- If the file doesn't have a valid magic number, then an error like "invalid file (bad magic number): Exec format error" occurs
However someone told me that if the file is plain text, then the shell tries to execute the commands (as if I had typed bash somefile
). I didn't believe this, but I just tried it, and it was correct. So I clearly have some misconceptions about what actually happens here, and would like to understand the mechanics.
What exactly happens when I execute a file in my shell? (in as much detail is reasonable…)
Best Answer
The definitive answer to "how programs get run" on Linux is the pair of articles on LWN.net titled, surprisingly enough, How programs get run and How programs get run: ELF binaries. The first article addresses scripts briefly. (Strictly speaking the definitive answer is in the source code, but these articles are easier to read and provide links to the source code.)
A little experimentation show that you pretty much got it right, and that the execution of a file containing a simple list of commands, without a shebang, needs to be handled by the shell. The execve(2) manpage contains source code for a test program, execve; we'll use that to see what happens without a shell. First, write a testscript,
testscr1
, containingand another one,
testscr2
, containing onlyMake them both executable, and verify that they both run from a shell:
Now try again, using
execve
(assuming you built it in the current directory):testscr1
still runs, buttestscr2
producesThis shows that the shell handles
testscr2
differently. It doesn't process the script itself though, it still uses/bin/sh
to do that; this can be verified by pipingtestscr2
toless
:On my system, I get
As you can see, there's the shell I was using,
zsh
, which startedless
, and a second shell, plainsh
(dash
on my system), to run the script, which ranpstree
. Inzsh
this is handled byzexecve
inSrc/exec.c
: the shell usesexecve(2)
to try to run the command, and if that fails, it reads the file to see if it has a shebang, processing it accordingly (which the kernel will also have done), and if that fails it tries to run the file withsh
, as long as it didn't read any zero byte from the file:bash
has the same behaviour, implemented inexecute_cmd.c
with a helpful comment (as pointed out by taliezin):POSIX defines a set of functions, known as the
exec(3)
functions, which wrapexecve(2)
and provide this functionality too; see muru's answer for details. On Linux at least these functions are implemented by the C library, not by the kernel.