Find two consecutive repeated lines

awkperlsedtext processing

how to find two concatenate repeated lines in files

for example in this file we have only two concatenate repeated lines

 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05.ear
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter <--
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter <--
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05.xml

Best Answer

Uniq should be enough:

$ cat c.txt
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05.ear
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05.xml

$ uniq -D c.txt
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter

$ uniq c.txt
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05.ear
 OQ-63/ECC/Global/MES/CZ/adWerum-CZ-Adapter
 OQ-63/ECC/Global/MES/54/ECC-MRP-S05.xml

By default uniq checks adjacent lines of the input file. So for an unsorted file (like your case) uniq will do the job you want.

you might also be interested in uniq -d and -u option. See man page for more details (-d prints only one of the both duplicate lines , -u print only uniq lines - removes both duplicate entries).

Related Question