Cut Command Not Working in Pipe – Troubleshooting Guide

cutgrep

I am able to filter out jobs which got stuck in our queueing system with:

> qjobs | grep "racon" 
       5240703 racon-3/utg001564l-racon-3.fasta  H   1     1   0     10.0   0.0    150   :03   
       5241418 racon-3/utg002276l-racon-3.fasta  H   1     1   0     10.0   0.0    150   :02   
       5241902 racon-3/utg002759l-racon-3.fasta  H   1     1   0     10.0   0.0    150   :03   
       5242060 racon-3/utg002919l-racon-3.fasta  H   1     1   0     10.0   0.0    150   :04   
       5242273 racon-3/utg003133l-racon-3.fasta  H   1     1   0     10.0   0.0    150   :03   
       5242412 racon-3/utg003270l-racon-3.fasta  H   1     1   0     10.0   0.0    150   :04   
       5242466 racon-3/utg003325l-racon-3.fasta  H   1     1   0     10.0   0.0    150   :03

However, qjobs | grep "racon" | cut -d " " -f2 did not return e.g. racon-3/utg003325l-racon-3.fasta. What did I miss?

Best Answer

Every space counts towards the field number, even leading and consecutive ones. Hence, you need to use -f9 instead of -f2. Alternatively, you can use awk '{ print $2 }' in place of the cut command entirely.

Related Solutions

Text Processing with Sed and Grep – Return Only the Portion of a Line After a Matching Pattern

The canonical tool for that would be sed.

sed -n -e 's/^.*stalled: //p'

Detailed explanation:

-n means not to print anything by default.
-e is followed by a sed command.
s is the pattern replacement command.
The regular expression ^.*stalled: matches the pattern you're looking for, plus any preceding text (.* meaning any text, with an initial ^ to say that the match begins at the beginning of the line). Note that if stalled: occurs several times on the line, this will match the last occurrence.
The match, i.e. everything on the line up to stalled:, is replaced by the empty string (i.e. deleted).
The final p means to print the transformed line.

If you want to retain the matching portion, use a backreference: \1 in the replacement part designates what is inside a group $…$ in the pattern. Here, you could write stalled: again in the replacement part; this feature is useful when the pattern you're looking for is more general than a simple string.

sed -n -e 's/^.*\(stalled: \)/\1/p'

Sometimes you'll want to remove the portion of the line after the match. You can include it in the match by including .*$ at the end of the pattern (any text .* followed by the end of the line $). Unless you put that part in a group that you reference in the replacement text, the end of the line will not be in the output.

As a further illustration of groups and backreferences, this command swaps the part before the match and the part after the match.

sed -n -e 's/^\(.*\)\(stalled: \)\(.*\)$/\3\2\1/p'

Pipe – How to Unbuffer Cut Command

If you're on a system using GNU Coreutils (almost any Linux), you can try stdbuf:

… | stdbuf -oL cut -d '@' -f 1 | …

-oL makes it line buffered, which seems like what you want.

Best Answer

Related Solutions

Text Processing with Sed and Grep – Return Only the Portion of a Line After a Matching Pattern

Pipe – How to Unbuffer Cut Command

Related Question