Shell – How to create a large file in UNIX

filesshell

I found a way in Windows to do such thing 

echo "This is just a sample line appended  to create a big file. " > dummy.txt
for /L %i in (1,1,21) do type dummy.txt >> dummy.txt

http://www.windows-commandline.com/how-to-create-large-dummy-file/

Is there a way in UNIX to copy a file, append and then repeat the process?
Something like for .. cat file1.txt > file1.txt?

Best Answer

yes "Some text" | head -n 100000 > large-file

With csh/tcsh:

repeat 10000 echo some test > large-file

With zsh:

{repeat 10000 echo some test} > large-file

On GNU systems, see also:

seq 100000 > large-file

Or:

truncate -s 10T large-file

(creates a 10TiB sparse file (very large but doesn't take any space on disk)) and the other alternatives discussed at "Create a test file with lots of zero bytes".

Doing cat file >> file would be a bad idea.

First, it doesn't work with some cat implementations that refuse to read files that are the same as their output file. But even if you work around it by doing cat file | cat >> file, if file is larger than cat's internal buffer, that would cause cat to run in an infinite loop as it would end up reading the data that it has written earlier.

On file systems backed by a rotational hard drive, it would be pretty inefficient as well (after reaching a size greater than would possibly be cached in memory) as the drive would need to go back and forth between the location where to read the data, and that where to write it.

Related Question