Ubuntu – Find & Replace in 80mb file

command linetext processing

Is there a command line or a tool in Ubuntu to "Find & Replace" a word in a 80mb file that accours about 150000 times?

I want to replace http://www.old-domain.com/subfolderA with http://www.new-domain.com/subfolderB

I tried it with gEdit and Atom but both crashed.

Best Answer

sed can do:

sed -i.bak '/oldword/s//newword/g' very_big_file

This edits the file directly leaving a backup named very_big_file.bak. It scans your file for lines containing oldword and replaces every occurence with newword, which should be much faster than running s/oldword/newword/g over every line (see Replace text quickly in very large file). Quoting sed1line:

OPTIMIZING FOR SPEED: If execution speed needs to be increased (due to large input files or slow processors or hard disks), substitution will be executed more quickly if the "find" expression is specified before giving the "s/.../.../" instruction. Thus:

sed 's/foo/bar/g' filename         # standard replace command   
sed '/foo/ s/foo/bar/g' filename   # executes more quickly
sed '/foo/ s//bar/g' filename      # shorthand sed syntax

If oldword and/or newword contain slashes you can either escape them with backslash (e.g. http:\/\/www) or use a different delimiter, e.g. underscore:

sed -i.bak '/oldword/s__newword_g' very_big_file
sed -i.bak '\_oldword_s//newword/g' very_big_file
sed -i.bak '\_oldword_s__newword_g' very_big_file

In your exact case I would do:

sed -i.bak '\_http://www.old-domain.com/subfolderA_s__http://www.new-domain.com/subfolderB_g' very_big_file
Related Question