Basic sed command on large one-line file: couldn’t re-allocate memory

large filesout of memoryperformancesedtext processing

I have a 250 MB text file, all in one line.

In this file I want to replace a characters with b characters:

sed -e "s/a/b/g" < one-line-250-mb.txt

It fails with:

sed: couldn't re-allocate memory

It seems to me that this kind of task could be performed inline without allocating much memory.
Is there a better tool for the job, or a better way to use sed?


GNU sed version 4.2.1
Ubuntu 12.04.2 LTS
1 GB RAM

Best Answer

Yes, use tr instead:

tr 'a' 'b' < file.txt > output.txt

sed deals in lines so a huge line will cause it problems. I expect it is declaring a variable internally to hold the line and your input exceeds the maximum size allocated to that variable.

tr on the other hand deals with characters and should be able to handle arbitrarily long lines correctly.

Related Question