How to delete all lines in a text file which have less than ‘x’ characters

awksedtext processing

How can I delete all lines in a text file which have fewer than 'x' letters OR numbers OR symbols? I can't use awk 'length($0)>' as it will include spaces.

Best Answer

Assuming you want to delete lines that contain less than n graphical symbols:

awk -v n=5 '{ line = $0; gsub("[^[:graph:]]", "") } length >= n { print line }'

This deletes all characters that does not match [[:graph:]]. If the length of the string that remains is greater than or equal to n, the (unmodified) line is printed.

The value of n is given on the command line.

[[:graph:]] is equivalent to [[:alnum:][:punct:]], which in turn is the same as [[:alpha:][:digit:][:punct:]]. It is roughly the same as [[:print:]] but does not match spaces.

Instead of [^[:graph:]], you could possibly use [[:blank:]] to delete all tabs or spaces.

With sed, following the above awk code almost literally,

sed -e 'h; s/[^[:graph:]]//g' \
    -e '/.\{5\}/!d; g'

or, simplified (only counting non-blank characters),

sed -e 'h; s/[[:blank:]]//g' \
    -e '/...../!d; g'

This first saves the current line into the hold space with h. It then deletes all non-graph characters (or blank characters in the second variation) on the line with s///g. If the line then contains less than 5 characters (change this to whatever number you want, or change the number of dots in the second variation), the line is deleted. Else, the stored line is fetched from the hold space with g and (implicitly) printed.

Related Solutions

How to remove lines shorter than XY

You could use sed. The following would remove lines that are 3 characters long or smaller:

sed -r '/^.{,3}$/d' filename

In order to save the changes to the file in-place, supply the -i option.

If your version of sed doesn't support extended RE syntax, then you could write the same in BRE:

sed '/^.\{,3\}$/d' filename

which would work with all sed variants.

You could also use awk:

awk 'length($0)>3' filename

Using perl:

perl -lne 'length()>3 && print' filename

Text Processing – How to Delete All Text Between Curly Brackets in a Multiline Text File

$ sed ':again;$!N;$!b again; s/{[^}]*}//g' file
This is 
that wants
 anyway.

Explanation:

:again;$!N;$!b again;

This reads the whole file into the pattern space.

:again is a label. N reads in the next line. $!b again branches back to the again label on the condition that this is not the last line.
s/{[^}]*}//g

This removes all expressions in braces.

On Mac OSX, try:

sed -e ':again' -e N -e '$!b again' -e 's/{[^}]*}//g' file

Nested Braces

Let's take this as a test file with lots of nested braces:

a{b{c}d}e
1{2
}3{
}
5

Here is a modification to handle nested braces:

$ sed ':again;$!N;$!b again; :b; s/{[^{}]*}//g; t b' file2
ae
13
5