Ubuntu – awk – How to print the number of characters for the first n lines in a file

It's ever-so clunky but you can add every line to an array and at the end —when you know the length— output everything but the last 3 lines.

... | awk '{l[NR] = $0} END {for (i=1; i<=NR-3; i++) print l[i]}'

Another (more efficient here) approach is manually stacking in three variables:

... | awk '{if (a) print a; a=b; b=c; c=$0}'

a only prints after a line has moved from c to b and then into a so this limits it to three lines. The immediate upsides are it doesn't store all the content in memory and it shouldn't cause buffering issues (fflush() after printing if it does) but the downside here is it's not simple to scale this up. If you want to skip the last 100 lines, you need 100 variables and 100 variable juggles.

If awk had push and pop operators for arrays, it would be easier.

Or we could pre-calculate the number of lines and how far we actually want to go with $(($(wc -l < file) - 3)). This is relatively useless for streamed content but on a file, works pretty well:

awk -v n=$(($(wc -l < file) - 3)) 'NR<n' file

Typically speaking you'd just use head though:

$ seq 6 | head -n-3
1
2
3

Using terdon's benchmark we can actually see how these compare. I thought I'd offer a full comparison though:

• head: 0.018s (me)
• awk + wc: 0.169s (me)
• awk 3 variables: 0.178s (me)
• awk double-file: 0.322s (terdon)
• awk circular buffer: 0.355s (Scrutinizer)
• awk for-loop: 0.693s (me)

The fastest solution is using a C-optimised utility like head or wc handle the heavy lifting things but in pure awk, the manually rotating stack is king for now.

This should do:

awk -F, '{x[$1]["count"]++;x[$1]["sum"]+=$2}END{for(y in x){print y,x[y]["count"],x[y]["sum"]}}' in

Basically you replace the array with a multidimensional array in order to store both the count of the occurences of each unique first field and the sum of their relative second fields.

% cat in
A,15
C,13
C,4
A,10
B,12
C,1
% awk -F, '{x[$1]["count"]++;x[$1]["sum"]+=$2}END{for(y in x){print y,x[y]["count"],x[y]["sum"]}}' in
A 2 25
B 1 12
C 3 18