How to treat a file as a single line with grep to apply a regexp search pattern

grepregular expression

I want to match everything that is between some lines with regexp but not this that match the start and the end. This sound to me as positive lookbehind and positive lookahead

    start text
    bla bla 
    bla
    end

There are multiple number of this kind of blocks so I would like to extract all this blocks and then for each of these blocks I would like to extract something based on a different regexp. So it should be something like:

match start 
then match everything until the first occurrence of end
match start 
then match everything until the first occurrence of end

and so on…

so I made something like this: (?<=start).*(?=end)

This does not work because I guess I use the command line grep, which treats the file as a set of lines and tries to apply the regex in every single line. Is there any way to treat the file as a whole line or this is not a good solution and I have to use a combination of various command line tools like extracting the text with sed and then build a file whose lines contains the concatenation of various lines from the initial file?

Best Answer

Since a'r beat me to the sed solution, I'll just post the perl equivalent:

perl -ne 'print if/start/../end/'

It's a bit more verbose though.

Related Solutions

Grep – How to Match Pattern Exactly from File and Search Only in First Column

You probably want the -wflag - from man grep

   -w, --word-regexp
          Select  only  those  lines  containing  matches  that form whole
          words.  The test is that the matching substring must  either  be
          at  the  beginning  of  the  line,  or  preceded  by  a non-word
          constituent character.  Similarly, it must be either at the  end
          of  the  line  or  followed by a non-word constituent character.
          Word-constituent  characters  are  letters,  digits,   and   the
          underscore.

i.e.

grep -wFf patfile file
denovo1 xxx yyyy oggugu ddddd
denovo22 hhhh yyyy kkkk iiii

To enforce matching only in the first column, you would need to modify the entries in the pattern file to add a line anchor: you could also make use of the \b word anchor instead of the command-line -w switch e.g. in patfile:

^denovo1\b
^denovo3\b
^denovo22\b

then

grep -f patfile file
denovo1 xxx yyyy oggugu ddddd
denovo22 hhhh yyyy kkkk iiii

Note that you must drop the -F switch if the file contains regular expressions instead of simple fixed strings.

Grep Command – How to Skip N Lines and Search After

With sed you can use a range and quit input at a single completion:

sed '/^182$/p;//,/^ABC$/!d;/^ABC$/!d;q'

Similarly w/ GNU grep you can split the input between two greps:

{ grep -nxF -m1 182; grep -nxF -m1 ABC; } <<\IN
123
XXY
214
ABC
182
558
ABC
856
ABC
IN

... which prints...

5:182
2:ABC

... to signify that the first grep found a -Fixed-string literal, -xentire-line 182 match 5 lines from the start of its read, and the second found a similarly typed ABC match 2 lines from the start of its read - or 2 lines after the first grep quit reading at line 5.

From man grep:

-m NUM, --max-count=NUM
          Stop  reading  a  file  after  NUM  matching
          lines.   If the input is standard input from
          a regular file, and NUM matching  lines  are
          output, grep ensures that the standard input
          is  positioned  to  just  after   the   last
          matching  line before exiting, regardless of
          the  presence  of  trailing  context  lines.
          This  enables  a calling process to resume a
          search.

I used a here-document for the sake of reproducible demonstration, but you should probably do:

{ grep ...; grep ...; } </path/to/log.file

It will also work with other shell compound-command constructs like:

for p in 182 ABC; do grep -nxFm1 "$p"; done </path/to/log.file

Best Answer

Related Solutions

Grep – How to Match Pattern Exactly from File and Search Only in First Column

Grep Command – How to Skip N Lines and Search After

Related Question