Linux – How Does It Deal with Shell Scripts?

linuxshellshell-script

For this question, let's consider a bash shell script, though this question must be applicable to all types of shell script.

When someone executes a shell script, does Linux load all the script at once (into memory maybe) or does it read script commands one by one (line by line)?

In other words, if I execute a shell script and delete it before the execution completes, will the execution be terminated or will it continue as it is?

Best Answer

If you use strace you can see how a shell script is executed when it's run.

Example

Say I have this shell script.

$ cat hello_ul.bash 
#!/bin/bash

echo "Hello Unix & Linux!"

Running it using strace:

$ strace -s 2000 -o strace.log ./hello_ul.bash
Hello Unix & Linux!
$

Taking a look inside the strace.log file reveals the following.

...
open("./hello_ul.bash", O_RDONLY)       = 3
ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff0b6e3330) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "#!/bin/bash\n\necho \"Hello Unix & Linux!\"\n", 80) = 40
lseek(3, 0, SEEK_SET)                   = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=4*1024}) = 0
fcntl(255, F_GETFD)                     = -1 EBADF (Bad file descriptor)
dup2(3, 255)                            = 255
close(3)     
...

Once the file's been read in, it's then executed:

...
read(255, "#!/bin/bash\n\necho \"Hello Unix & Linux!\"\n", 40) = 40
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc0b38ba000
write(1, "Hello Unix & Linux!\n", 20)   = 20
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
read(255, "", 40)                       = 0
exit_group(0)                           = ?

In the above we can clearly see that the entire script appears to be being read in as a single entity, and then executed there after. So it would "appear" at least in Bash's case that it reads the file in, and then executes it. So you'd think you could edit the script while it's running?

NOTE: Don't, though! Read on to understand why you shouldn't mess with a running script file.

What about other interpreters?

But your question is slightly off. It's not Linux that's necessarily loading the contents of the file, it's the interpreter that's loading the contents, so it's really up to how the interpreter's implemented whether it loads the file entirely or in blocks or lines at a time.

So why can't we edit the file?

If you use a much larger script however you'll notice that the above test is a bit misleading. In fact most interpreters load their files in blocks. This is pretty standard with many of the Unix tools where they load blocks of a file, process it, and then load another block. You can see this behavior with this U&L Q&A that I wrote up a while ago regarding grep, titled: How much text does grep/egrep consume each time?.

Example

Say we make the following shell script.

$ ( 
    echo '#!/bin/bash'; 
    for i in {1..100000}; do printf "%s\n" "echo \"$i\""; done 
  ) > ascript.bash;
$ chmod +x ascript.bash

Resulting in this file:

$ ll ascript.bash 
-rwxrwxr-x. 1 saml saml 1288907 Mar 23 18:59 ascript.bash

Which contains the following type of content:

$ head -3 ascript.bash ; echo "..."; tail -3 ascript.bash 
#!/bin/bash
echo "1"
echo "2"
...
echo "99998"
echo "99999"
echo "100000"

Now when you run this using the same technique above with strace:

$ strace -s 2000 -o strace_ascript.log ./ascript.bash
...    
read(255, "#!/bin/bash\necho \"1\"\necho \"2\"\necho \"3\"\necho \"4\"\necho \"5\"\necho \"6\"\necho \"7\"\necho \"8\"\necho \"9\"\necho \"10\"\necho 
...
...
\"181\"\necho \"182\"\necho \"183\"\necho \"184\"\necho \"185\"\necho \"186\"\necho \"187\"\necho \"188\"\necho \"189\"\necho \"190\"\necho \""..., 8192) = 8192

You'll notice that the file is being read in at 8KB increments, so Bash and other shells will likely not load a file in its entirety, rather they read them in in blocks.

References

The #! magic, details about the shebang/hash-bang mechanism on various Unix flavours

Related Solutions

Linux – What Happens If You Edit a Script During Execution?

In Unix, most editors work by creating a new temporary file containing the edited contents. When the edited file is saved, the original file is deleted and the temporary file renamed to the original name. (There are, of course, various safeguards to prevent dataloss.) This is, for example, the style used by sed or perl when invoked with the -i ("in-place") flag, which is not really "in-place" at all. It should have been called "new place with old name".

This works well because unix assures (at least for local filesystems) that an opened file continues to exist until it is closed, even if it is "deleted" and a new file with the same name is created. (It's not coincidental that the unix system call to "delete" a file is actually called "unlink".) So, generally speaking, if a shell interpreter has some source file open, and you "edit" the file in the manner described above, the shell won't even see the changes since it still has the original file open.

[Note: as with all standards-based comments, the above is subject to multiple interpretations and there are various corner-cases, such as NFS. Pedants are welcome to fill the comments with exceptions.]

It is, of course, possible to modify files directly; it's just not very convenient for editing purposes, because while you can overwrite data in a file, you cannot delete or insert without shifting all following data, which would imply quite a lot of rewriting. Furthermore, while you were doing that shifting, the contents of the file would be unpredictable and processes which had the file open would suffer. In order to get away with this (as with database systems, for example), you need a sophisticated set of modification protocols and distributed locks; stuff which is well beyond the scope of a typical file editing utility.

So, if you want to edit a file while its being processed by a shell, you have two options:

You can append to the file. This should always work.
You can overwrite the file with new contents of exactly the same length. This may or may not work, depending on whether the shell has already read that part of the file or not. Since most file I/O involves read buffers, and since all the shells I know read an entire compound command before executing it, it is pretty unlikely that you can get away with this. It certainly wouldn't be reliable.

I don't know of any wording in the Posix standard which actually requires the possibility of appending to a script file while the file is being executed, so it might not work with every Posix compliant shell, much less with the current offering of almost- and sometimes-posix-compliant shells. So YMMV. But as far as I know, it does work reliably with bash.

As evidence, here's a "loop-free" implementation of the infamous 99 bottles of beer program in bash, which uses dd to overwrite and append (the overwriting is presumably safe because it substitutes the currently executing line, which is always the last line of the file, with a comment of exactly the same length; I did that so that the end result can be executed without the self-modifying behaviour.)

#!/bin/bash
if [[ $1 == reset ]]; then
  printf "%s\n%-16s#\n" '####' 'next ${1:-99}' |
  dd if=/dev/stdin of=$0 seek=$(grep -bom1 ^#### $0 | cut -f1 -d:) bs=1 2>/dev/null
  exit
fi

step() {
  s=s
  one=one
  case $beer in
    2) beer=1; unset s;;
    1) beer="No more"; one=it;;
    "No more") beer=99; return 1;;
    *) ((--beer));;
  esac
}
next() {
  step ${beer:=$(($1+1))}
  refrain |
  dd if=/dev/stdin of=$0 seek=$(grep -bom1 ^next\  $0 | cut -f1 -d:) bs=1 conv=notrunc 2>/dev/null
}
refrain() {
  printf "%-17s\n" "# $beer bottles"
  echo echo ${beer:-No more} bottle$s of beer on the wall, ${beer:-No more} bottle$s of beer.
  if step; then
    echo echo Take $one down, pass it around, $beer bottle$s of beer on the wall.
    echo echo
    echo next abcdefghijkl
  else
    echo echo Go to the store, buy some more, $beer bottle$s of beer on the wall.
  fi
}
####
next ${1:-99}   #

Bash – Prompt user to login as root when running a shell script

This is very easy to accomplish:

#!/bin/sh
[ "$(whoami)" != "root" ] && exec sudo -- "$0" "$@"

When the current user isn't root, re-exec the script through sudo.

Note that I am using sudo here instead of su. This is because it allows you to preserve arguments. If you use su, your command would have to be su -c "$0 $@" which would mangle your arguments if they have spaces or special shell characters.

If your shell is bash, you can avoid the external call to whoami:

(( EUID != 0 )) && exec sudo -- "$0" "$@"

Best Answer

Example

What about other interpreters?

So why can't we edit the file?

Example

References

Related Solutions

Linux – What Happens If You Edit a Script During Execution?

Bash – Prompt user to login as root when running a shell script

Related Question