text-processing csv – How to Cut Every 100th Column from CSV

csvtext processing

I have a data file of numbers seperated by tabs, like this

1 2 3 4
2 4 6 8

My real file is 50000 columns wide and I only need every 100th column (column 100, 200, 300, 400, …).
Now I would like to remove all the other columns.

How can I do that?

Best Answer

That's what awk is for:

awk '{for(i=100;i<=NF;i+=100){printf "%s ",$i;} print ""}' file > output

Or, if you can have spaces inside your fields, specify tab as the field separator:

awk -F'\t' '{for(i=100;i<=NF;i+=100){printf "%s ",$i;} print ""}' file > output

Alternatively, you could use Perl:

perl -ane 'for($i=99;$i<=$#F;$i+=100){print "$F[$i] "}' file > output

To do this for multiple files, you can use a shell loop (assuming you want to run this on all files in the current directory):

for f in *; do
  awk '{for(i=100;i<=NF;i+=100){printf "%s ",$i;} print ""}' "$f" > "$f".new;
done
Related Question