Passing regular files only to `sed -i`

filessedwildcards

I am using GNU sed 4.2.2 and after searching cannot find out the reason sed behaves oddly in some situations:

I have a directory with the following:

foofile.txt
barfile.txt
bazfile.txt
config/

Case 1

sed -i 's/foo/bar/g' *.txt

This works as I expect. It replaces all the "foo"s with "bar"s in the three regular files.

Case 2

sed -i 's/foo/bar/g' *
sed: couldn't edit config: not a regular file

It replaces "foo" with "bar" in barfile.txt and bazfile.txt, but not in foofile.txt. I assume it goes through the list of files expanded from the * alphabetically, and when it hits config/ it errors and exits. Is there a way to have sed ignore errors and continue processing files?

Case 3

for file in $(find . -maxdepth 1 -type f); do sed -i 's/foo/bar/g' <"$file"; done
sed: no input files
sed: no input files
sed: no input files

Could someone please explain why sed does this? Why does it say there's no input file when it's being given one?

I know that I can use the following but I'm asking why sed acts this way, not how do I solve this one use case.

find . -maxdepth 1 -type f -exec sed -i 's/foo/bar/g' {} \;

Best Answer

It's normal behavior. In both cases sed exits with error code 4... per info sed:

4
     An I/O error, or a serious processing error during runtime,
     GNU 'sed' aborted immediately.

and in both cases the messages are self-explanatory. Not sure what's unclear but for the record: the first time it errors out because it cannot edit a directory and the second time it complains because it cannot edit stdin in-place, it needs a file (i.e. remove that redirect before $file)
The proper way to do this with find is, as you noted, via -exec ...
With globs, you'll have to use a loop and test if input is a regular file before running sed. Or, if you're a zsh user, you can simply do:

sed -i 's/foo/bar/g' *(.)

Related Solutions

Python – case sensitive substitution; same target ids

You can't really do this kind of thing with sed, it's just a text stream editor. Try this Perl scriptlet:

#!/usr/bin/env perl 

## Set the record separator to \n\n to
## read multiple lines as a single record
$/="\n\n";
## This array will contain all lines of the file
my @lines=<>;

## The list of suffixes
@suffix=(a..z); 

## For each line of the input file
foreach (@lines) {
    ## If the current line (lines are now the actual multiline records
    ## because we set $/ to consecutive newlines) is one we are interested in.
    if (/isoforms.*?Target=(\S+)/s){
    ## Keep a list of seen targets
    $seen{$1}++;
    }

}
## Now that we have processed the entire file
## go back and print each line.
foreach (@lines) {

    ## If this line is one of the ones we're interested in
    if(/Name=(.+?);.*?isoforms=.*?Target=(\S+)/s){
    $name=$1; $target=$2;
    ## This is needed so we can know whether
    ## how many times we've seen this target so far.
    $newseen{$target}++;
    ## If this target exists more than once in the input file
    if ($seen{$target}>1) {
        ## Use the %newseen hash to choose the right letter.
        ## The -1 is needed because the first element of an
        ## array is 0, not 1.
        s/$name/$target.$suffix[$newseen{$target}-1]/;
    }
    else {
        s/$name/$target/;
    }
    }
    print;
}

Save the script above as foo.pl, make it executable (chmod a+x foo.pl) and run on your input file:

./foo.pl input.txt > output.txt

How to replace characters only on certain lines

The complication here is that if you replace all ' with " and then all " with ', you will be left with only '. So, you will first need to replace ' with something else—for example, the NULL character (\0) which you can safely assume won't be in your input file—then " with ' and then replace that something else with " again. For example, in perl:

$ perl -pe "if(/KEYWORD){s/'/\0/g; s/\"/'/g; s/\0/\"/g}" file
KEYWORD_1 table name column = 'string' AND column = 'string'
Additional text
      .
      .
      .
KEYWORD_2 text "text in quote" etc.

Explanation

-pe : print each input line after applying the script given by -e.
if(/KEYWORD/){something} : do something only if this line matches KEYWORD.
s/foo/bar/g : replace all occurrences of foo in the line with bar. The g means "global". Without it, only the 1st occurrence on each line would be replaced.

Note that since the script itself is inside double quotes, the double quotes inside the script need to be escaped (\").

As pointed out in the comments, there's a more direct way I should have thought of in the first place:

perl -pe "tr/'\"/\"'/ if /^KEYWORD/" file

The tr operator transliterates lists of characters. The general format is tr/searchlist/replacementlist/. So this will replace all ' with " and all " with ' only on lines matching KEYWORD.