Ubuntu – How to add the text before the line in whole file

Using awk

awk '!x[$0]++' infile.txt > outfile.txt

the way it works is that it keeps count of the lines in an array, and if the current count is zero, ie the first occurance, it prints the line, otherwise it continues to the next one.

• Using awk, printing first and last field with \n as delimiter:

  awk '{printf "%s\n%s\n", $1, $NF}' file.txt
  

• Using sed, capturing first and last field while matching and using in replacement:

  sed -E 's/([^[:blank:]]+).*[[:blank:]]([^[:blank:]]+)$/\1\n\2/' file.txt
  

• With perl, similar logic to sed:

  perl -pe 's/^([^\s]+).*\s([^\s]+)/$1\n$2/' file.txt
  

• Using bash, slower approach, creating an array from each line and printing first and last element from the array separating them by \n:

  while read -ra line; do printf '%s\n%s\n' "${line[0]}" \
         "${line[$((${#line[@]]}-1))]}"; done <file.txt
  

• With python, creating a list containing whitespace separated elements from each line, then printing the first and last element from the list, separating by \n:

  #!/usr/bin/env python3
  with open("file.txt") as f:
      for line in f:
          line = line.split()
          print(line[0]+'\n'+line[-1])
  

Example:

$ cat file.txt                               
UniRef90_Q8YC41 Putative binding protein BMEII0691 MNRFIAFFRSVFLIGLVATAFGRACA
UniRef90_Q8YC41 Putative binding protein BMEII0691 MNRFIAFFRSVFLIGLVATAFGRACA

$ awk '{printf "%s\n%s\n", $1, $NF}' file.txt                             
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA

$ sed -E 's/([^[:blank:]]+).*[[:blank:]]([^[:blank:]]+)$/\1\n\2/' file.txt
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA

$ perl -pe 's/^([^\s]+).*\s([^\s]+)/$1\n$2/' file.txt
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA


$ while read -ra line; do printf '%s\n%s\n' "${line[0]}" "${line[$((${#line[@]]}-1))]}"; done <file.txt
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA

>>> with open("file.txt") as f:
...     for line in f:
...         line = line.split()
...         print(line[0]+'\n'+line[-1])
... 
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA
UniRef90_Q8YC41
MNRFIAFFRSVFLIGLVATAFGRACA