Efficient Way to Change One Line in a File

filesfindperformanceshell-script

I want to change the first line of hundreds of files recursively in the most efficient way possible. An example of what I want to do is to change #!/bin/bash to #!/bin/sh, so I came up with this command:

find ./* -type f -exec sed -i '1s/^#!\/bin\/bash/#!\/bin\/sh/' {} \;

But, to my understanding, doing it this way sed has to read the whole file and replace the original. Is there a more efficient way to do this?

Best Answer

Yes, sed -i reads and rewrites the file in full, and since the line length changes, it has to, as it moves the positions of all other lines.

...but in this case, the line length doesn't actually need to change. We can replace the hashbang line with #!/bin/sh␣␣ instead, with two trailing spaces. The OS will remove those when parsing the hashbang line. (Alternatively, use two newlines, or a newline + hash sign, both of which create extra lines the shell will eventually ignore.)

All we need to do is to open the file for writing from the start, without truncating it. The usual redirections > and >> can't do that, but in Bash, the read-write redirection <> seems to work:

echo '#!/bin/sh  ' 1<> foo.sh

or using dd (these should be standard POSIX options):

echo '#!/bin/sh  ' | dd of=foo.sh conv=notrunc

Note that strictly speaking, both of those rewrite the newline at the end of the line too, but it doesn't matter.

Of course, the above overwrites the start of the given file unconditionally. Adding a check that the original file has the correct hashbang is left as an exercise... Regardless, I probably wouldn't do this in production, and obviously, this won't work if you need to change the line to a longer one.

Related Question