Shell – Randomly draw a certain number of lines from a data file

linuxshelltext processing

I have a data list, like

Assume the size of this data set (i.e. lines of file) is N. I want to randomly draw m lines from this data file. Therefore, the output should be two files, one is the file including these m lines of data, and the other one includes N-m lines of data.

Is there a way to do that using a Linux command?

Best Answer

This might not be the most efficient way but it works:

shuf <file> > tmp
head -n $m tmp > out1
tail -n +$(( m + 1 )) tmp > out2

With $m containing the number of lines.

Related Solutions

Extract data from two files

All you need is join

join -t\, <(sort Output1.csv) <(sort Output2.csv)
-or-
join -t "," <(sort Output1.csv) <(sort Output2.csv)

or awk

awk -F, 'FNR==NR{a[$1]=$2;next}{ print $0 "," a[$1]}' Output2.csv Output1.csv

How to use awk to extract data from a file based on the content of another file

You can set variables like RS separately for each file. For example:

$ awk 'NR==FNR{a[$0];next} $1 in a' RS='\r?\n' title_list.txt RS='+++\r?\n' FS='\r?\n' ORS='+++\n' data.txt
article 1 title
article 1 body line 1
article 1 body line 2
+++
article 3 title
article 3 body line 1
article 3 body line 2
+++
article 4 title
article 4 body line 1
article 4 body line 2
article 4 body line 3
+++

Best Answer

Related Solutions

Extract data from two files

How to use awk to extract data from a file based on the content of another file

Related Question