Ubuntu – Get the unique lines of second file in result of comparing two files

bashcommand linetext processing

I have two text files, and I want to read file1 line by line, searching for the same line in file2 and removing it from file2.

I have the pseudocode of:

for line in file1.txt
do
  sed search line and delete in file2.txt
done

Best Answer

You could accomplish this with grep.

Here is an example:

$ echo localhost > local_hosts

$ grep -v -f local_hosts /etc/hosts
127.0.1.1       ubuntu

# The following lines are desirable for IPv6 capable hosts
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Related Solutions

Ubuntu – How to replace a string on the 5th line of multiple text files

Here are a few approaches. I am using brace expansion (file{1..4}.txt) which means file1.txt file2.txt file3.txt file4.txt

Perl
```
perl -i -pe 's/.*/ Good Morning / if $.==5' file{1..4}.txt
```
Explanation:
- -i: causes perl to edit the files in place, changing the original file.
  
  If -i is followed with a file extension suffix, then a backup is created for every file that is modified. Ex: -i.bak creates a file1.txt.bak if file1.txt is modified during the execution.
- -p: means read the input file line by line, apply the script and print it.
- -e: allows you to pass a script from the command line.
- s/.*/ Good Morning /: That will replace the text in the current line (.*) with Good Morning.
- $. is a special Perl variable that holds the current line number of the input file. So, s/foo/bar/ if $.==5, means replace foo with bar only on the 5th line.
sed
```
sed -i '5s/.*/ Good Morning /' file{1..4}.txt
```
Explanation:
- -i: Like for perl, edit file in place.
By default, sed prints each line of the input file. The 5s/pattern/replacement/ means substitute pattern with replacement on the 5th line.
Awk
```
for f in file{1..4}.txt; do 
    awk 'NR==5{$0=" Good Morning "}1;' "$f" > foobar && mv foobar "$f"; 
done
```
Explanation:

awk has no equivalent to the -i option¹ which means that we need to create a temporary file (foobar) which is then renamed to overwrite the original. The bash loop for f in file{1..4}.txt; do ... ; done simply goes through each of file{1..4}.txt, saving the current file name as $f. In awk, NR is the current line number and $0 is the content of the current line. So, the script will replace the line ($0) with " Good Morning " only on the 5th line. 1; is awk for "print the line".

¹_{Newer versions do as devnull showed in his answer.}
coreutils
```
for f in file{1..4}.txt; do 
    (head -4 "$f"; echo " Good Morning "; tail -n +6 "$f") > foobar && 
    mv foobar "$f"; 
done 
```
Explanation:

The loop is explained in the previous section.
- head -4: print the first 4 lines
- echo " Good Morning ": print " Good Morning "
- tail -n +6: print everything from the 6th line to the end of the file
The parentheses ( ) around those three commands allow you to capture the output of all three (so, 1st 4 lines, then " Good morning ", then the rest of the lines) and redirect them to a file.

Ubuntu – Comparing two text files

Try this command:

 grep -v -f file2.csv file1.csv > file3.csv

According to grep manual:

  -f FILE, --file=FILE
          Obtain  patterns  from  FILE,  one  per  line.   The  empty file
          contains zero patterns, and therefore matches nothing.   (-f  is
          specified by POSIX.)

  -v, --invert-match
          Invert the sense of matching, to select non-matching lines.  (-v
          is specified by POSIX.)

As Steeldriver said in his comment is better add also -x and -F that:

  -F, --fixed-strings
          Interpret PATTERN as a  list  of  fixed  strings,  separated  by
          newlines,  any  of  which is to be matched.  (-F is specified by
          POSIX.)
  -x, --line-regexp
          Select  only  those  matches  that exactly match the whole line.
          (-x is specified by POSIX.)

So, better command is:

 grep -xvFf file2.csv file1.csv > file3.csv

This command use file2.csv line as pattern and print line of file1.csv that doesn't match (-v).

Best Answer

Related Solutions

Ubuntu – How to replace a string on the 5th line of multiple text files

Explanation:

Explanation:

Explanation:

Explanation:

Ubuntu – Comparing two text files

Related Question