How to Remove a Paragraph from a File

bashsedshelltext processing

I have a file (file.php) like this:

...
Match user foo
        ChrootDirectory /NAS/foo.info/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no



Match user bar
        ChrootDirectory /NAS/bar.co.uk/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no



Match user baz
        ChrootDirectory /NAS/baz.com/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no

I am trying to write a bash script to delete one of the paragraphs.

So say I wanted delete the user foo from the file.php. After running the script, it would then look like this:

...
Match user bar
        ChrootDirectory /NAS/bar.co.uk/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no



Match user baz
        ChrootDirectory /NAS/baz.com/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no

How could I go about doing this. I have thought about using sed but that only seems to be appropriate for one liners?

sed -i 's/foo//g' file.php

And I couldn't do it for each individual line as most of the lines withing the paragraph are not unique! Any ideas?

Best Answer

Actually, sed can also take ranges. This command will delete all lines between Match user foo and the first empty line (inclusive):

$ sed '/Match user foo/,/^\s*$/{d}' file


Match user bar
        ChrootDirectory /NAS/bar.co.uk/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no



Match user baz
        ChrootDirectory /NAS/baz.com/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no

Personally, however, I would do this using perl's paragraph mode (-00) that has the benefit of removing the leading blank lines:

$ perl -00ne 'print unless /Match user foo/' file
Match user bar
        ChrootDirectory /NAS/bar.co.uk/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no

Match user baz
        ChrootDirectory /NAS/baz.com/
        ForceCommand internal-sftp
        AllowTcpForwarding no
        GatewayPorts no
        X11Forwarding no

In both cases, you can use -i to edit the file in place (these will create a backup of the original called file.bak):

sed -i.bak '/Match user foo/,/^\s*$/{d}' file

perl -i.bak -00ne 'print unless /Match user foo/' file

Related Solutions

Using sed to remove string or paragraph between delimiter

Doing this for single line strings is very simple:

sed 's/((([^)]*)))//g' file

If you need it to deal with multiline strings, it gets more complex. One approach would be to use tr to replace all newlines with the null character (\0), make the substitution and translate back again:

tr '\n' '\0' < file | sed 's/((([^)]*)))//g' | tr '\0' '\n'

Alternatively, you could just use perl:

perl -0pe 's/\(\(\([^)]+\)\)\)//g;' file

The -0 causes perl to read the entire file into memory (this might be a problem for huge files), the -p means "print each line" but because of the -0, the "line" is actually the entire file. The s/// is the same idea as for sed.

Shell – Extract paragraph separated with *** using AWK

Tell awk to print between the two delimiters. Specifically:

awk '/\*{4,}/,/<np>/' file

That will also print the lines containing the delimiters, so you can remove them with:

awk '/\*{4,}/,/<np>/' file | tail -n +2 | head -n -1

Alternatively, you can set a variable to true if a line matches the 1st delimiter and to false when it matches the second and only print when it is true:

awk '/\*{4,}/{a=1; next}/<np>/{a=0}(a==1){print}' file

The command above will set a to 1 if the current line matches 4 or more * and will also skip to the next line. This means that the *** line will never be printed.

This was in answer to the original, misunderstood, version of the question. I'm leaving it here since it can be useful in a slightly different situation.

First of all, you don't want FS (field separator), you want RS (record separator). Then, to pass a literal *, you need to escape it twice. Once to escape the * and once to escape the backslash (otherwise, awk will try to match it in the same way as \r or \t). Then, you print the 2nd "line":

$ awk -vRS='\\*\\*\\*' 'NR==2' file

thingsIwantToRead1   
thingsIwantToRead2   
thingsIwantToRead3

To avoid the blank lines around the output, use:

$ awk -vRS='\n\\*\\*\\*\n' 'NR==2' file
thingsIwantToRead1   
thingsIwantToRead2   
thingsIwantToRead3

Note that this assumes a *** after each paragraph, not only after the first one as you show.

Best Answer

Related Solutions

Using sed to remove string or paragraph between delimiter

Shell – Extract paragraph separated with *** using AWK

Related Question