Ubuntu – Asterisk not working with grep

That's the trick: command doesn't know, it's the shell that does the job

Consider for example grep 'abc' *.txt. If we run trace of system calls, you will see something like this:

bash-4.3$ strace -e trace=execve grep "abc" *.txt > /dev/null
execve("/bin/grep", ["grep", "abc", "ADDA_converters.txt", "after.txt", "altera_license.txt", "altera.txt", "ANALOG_DIGITAL_NOTES.txt", "androiddev.txt", "answer2.txt", "answer.txt", "ANSWER.txt", "ascii.txt", "askubuntu-profile.txt", "AskUbuntu_Translators.txt", "a.txt", "bash_result.txt", ...], [/* 80 vars */]) = 0
+++ exited with 0 +++

The shell expanded *.txt into all filenames in current directory that end with .txt extension. So effectively, your shell translates the grep 'abc' *.txt command into grep 'abc' file1.txt file2.txt file3.txt . . .. Thus, your second assumption is correct.

First assumption is not correct - programs have no way of detecting glob. It is possible to pass * as string argument to command, but it's the command's job to decide what to do with it then. Filename expansion, however, is property of your respective shell as I've already mentioned.

However, this only works because grep accepts multiple files at the end. It is possible that another command would only allow 1 file to be passed in.

Exactly right ! Programs don't limit the number of acceptable command-line arguments (for instance , in C that's array of strings const char *args[] and in python sys.argv[] ), but they can detect the length of that array or whether or not something unexpected is in wrong array position. grep doesn't do that, and accepts multiple files, which is by design.

On side note , improper quoting coupled with globbing with grep can sometimes be a problem. Consider this:

bash-4.3$ echo "one two" | strace -e trace=execve grep *est*
execve("/bin/grep", ["grep", "self_test.sh", "test.wxg"], [/* 80 vars */]) = 0
+++ exited with 1 +++

Unprepared user would expect that grep will match any line with est letters in it coming from pipe, but instead shell's filename expansion twisted everyting around. I've seen this happen a lot with people who do ps aux | grep shell_script_name.sh, and they expect to find their process running, but because they ran command from same directory where script was, shell's filename expansion made grep command to look completely different behind the scenes from what user expected.

Proper way would be to use single quotes:

bash-4.3$ echo "one two" | strace -e trace=execve grep '*est*'
execve("/bin/grep", ["grep", "*est*"], [/* 80 vars */]) = 0
+++ exited with 1 +++

Try this,

grep -f file1 file2 | awk '!_[$2]++'

 1  C95696352 score:  -69.785 nathvy =  38 nconfs =          888
 3  C96209347 score:  -57.128 nathvy =  24 nconfs =         1188
 6  C89372772 score:  -55.728 nathvy =  22 nconfs =         3228
20  C85594749 score:  -49.780 nathvy =  44 nconfs =         4536

Explanation

• grep -f file1 file2: search in file2 for matches of patterns obtained from file1
• awk '!_[$2]++': Don't print anything if field $2 has been seen before (via)
  • _ is the array name (can be anything, e.g. "seen")
  • _[$2]++ will create an array entry with the key being the content of field $2 and add 1
  • If _[$2] was not (!) already set, print the line. The printcommand is the default action that is made by awk when the condition matches.