Counting lines without text in a file

I am using Ubuntu 13.10 for a while now, programming in C++ and Python. I have some question that I can solve by making a program, but maybe there are easier ways, so here is the first question.

I can use grep to find lines with a word/pattern and then use wc to count them:

grep word somefile | wc -l

How do I count the lines that have no "text"? Those that are empty or have only spaces or tabs.

Best Answer

Your system should have GNU grep, that has an option -P to use Perl expressions and you can use that, combined with -c (so no need for wc -l):

grep -Pvc '\S' somefile

The '\S' hands the pattern \S to grep and matches all line containing anything that is not space, -v selects all the other lines (those only with space), and -c counts them.

From the man page for grep:

-P, --perl-regexp
       Interpret  PATTERN  as  a  Perl  regular  expression  (PCRE, see
       below).  This is highly experimental and grep  -P  may  warn  of
       unimplemented features.

-v, --invert-match
       Invert the sense of matching, to select non-matching lines.  (-v
       is specified by POSIX.)

-c, --count
       Suppress normal output; instead print a count of matching  lines
       for  each  input  file.  With the -v, --invert-match option (see
       below), count non-matching lines.  (-c is specified by POSIX.)

Related Solutions

Shell – Counting the characters of each line with wc

wc counts over the whole file; You can use awk to process line by line (not counting the line delimiter):

echo -e "foo\nbar\nbazz\n" | grep ba | awk '{print length}'

or as awk is mostly a superset of grep:

echo -e "foo\nbar\nbazz\n" | awk '/ba/ {print length}'

(note that some awk implementations report the number of bytes (like wc -c) as opposed to the number of characters (like wc -m) and others will count bytes that don't form part of valid characters in addition to the characters (while wc -m would ignore them in most implementations))

Way to count the number of lines of text in a file including non-delimited ones

With GNU sed, you can use:

sed '$=;d'

As GNU sed does consider those extra characters after the last newline as an extra line. GNU sed like most GNU utilities also supports NUL characters in its input and doesn't have a limitation on the length of lines (the two other criteria that make an input non-text as per POSIX).

POSIXLy, building-up on @Inian's answer to support too-long lines and NUL bytes:

LC_ALL=C tr -cs '\n' '[x*]' | awk 'END {print NR}'

That tr command translates all sequences of one or more character (each byte interpreted as a character in the C locale to avoid decoding issues) other than newline to one x character, so awk input records will be either 0 or 1 byte long and its input contain only x and newline characters.

$ printf '%10000s\na\0b\nc\nd' | wc -l
3

$ printf '%10000s\na\0b\nc\nd' | mawk 'END{print NR}'
2
$ printf '%10000s\na\0b\nc\nd' | busybox awk 'END{print NR}'
5
$ printf '%10000s\na\0b\nc\nd' | gawk 'END{print NR}'
4

$ printf '%10000s\na\0b\nc\nd' | LC_ALL=C tr -cs '\n' '[x*]' | mawk 'END{print NR}'
4

Best Answer

Related Solutions

Shell – Counting the characters of each line with wc

Way to count the number of lines of text in a file including non-delimited ones

Related Question