shell-script shell interpreter – Parsing Scripts at Runtime: Ubiquitous to Shells or Other Interpreters?

interpretershellshell-script

I had always thought that shells parse whole scripts, constructing an AST, and then execute that AST from memory. However, I just read a comment by Stéphane Chazelas, and tested executing this script, edit-while-executing.sh:

#!/bin/bash

echo start
sleep 10

and then while it was sleeping running:

$ echo "echo end" >> edit-while-executing.sh

and it worked to cause it to print "end" at the end.

However, when trying to modify this:

#!/bin/bash

while true; do
  echo yes
done

by doing:

$ printf "%s" "no " | dd of=edit-while-executing.sh conv=notrunc seek=35 bs=1

It didn't work, and kept printing "yes".

I also wondered if other non-shell interpreters also worked like this, and tried the equivalent of the first script with python, but it didn't work. Though, maybe python is not an interpreter anymore and it's more of a JIT compiler.

So to reiterate my question, is this a behaviour ubiquitous to shells and limited to them or also present in other interpreters (those not regarded as shells)? Also how does this work such that could I do the first modification but not the second?

Best Answer

So, this runs indefinitely in Bash/dash/ksh/zsh (or at least until your disk fills up):

#!/bin/sh
s=$0
foo() { echo "hello"; echo "foo" >> $s; sleep .1; }
foo

The thing to note, is that only stuff appended added to the script file after the last line the shell has read matters. The shells don't go back to re-read the earlier parts, which they even couldn't do, if the input was a pipe.

The similar construct doesn't work in Perl, it reads the whole file in before running.

#!/usr/bin/perl -l    
open $fd, ">>", $0;
sub foo { print "hello"; print $fd 'foo;' }
foo;

We can see that it does so also when given input through a pipe. This gives a syntax error (and only that) after 1 second:

$ (echo 'printf "hello\n";' ; sleep 1 ; echo 'if' ) | perl

While the same script piped to e.g. Bash, prints hello, and then throws the syntax error one second later.

Python appears similar to Perl with piped input, even though the interpreter runs a read-eval-print loop when interactive.

In addition to reading the input script line-by-line, at least Bash and dash process arguments to eval one line at a time:

$ cat evaltest.sh
var='echo hello
fi'
eval "$var"
$ bash evaltest.sh
hello
evaltest.sh: eval: line 4: syntax error near unexpected token `fi'
evaltest.sh: eval: line 4: `fi'

Zsh and ksh give the error immediately.

Similarly for sourced scripts, this time Zsh also runs line-by-line, as do Bash and dash:

$ cat sourceme.sh
echo hello
fi
$ zsh -c '. ./sourceme.sh'
hello
./sourceme.sh:2: parse error near `fi'
Related Question