Get top-level directory from makefile variable

That behaviour is not a bug. It is a feature. A feature and a possible user error to be precise.

The feature in question is one of the implicit rules of Make. In your case the implicit rule to "build" *.sh files. The user error, your error, is not changing the working directory before invoking the makefile in the subdirectories.

TL; DR: to fix this you can do one or more of the following:

1. Fix the shell script to change the working directory:

   #!/bin/bash
   
   for f in *; do
           if [[ -d $f && -f $f/makefile ]]; then
                   echo "Making clean in $f..."
                   (cd $f; make clean)
           fi
   done
   

2. Make the empty rules explicit:

   clean: ;
   

3. Make the clean targets phony:

   .PHONY: clean
   

Detailed explanation:

Make has a bunch of implicit rules. This allows one to invoke make on simple projects without even writing a makefile.

Try this for a demonstration:

1. create an empty directory and change in to the directory
2. create a file named clean.sh.
3. run make clean

Output:

$ make clean
cat clean.sh >clean 
chmod a+x clean

BAM! That is the power of implict rules of make. See the make manual about implicit rules for more information.

I will try to answer the remaining open question:

Why does it not invoke the implicit rule for the first makefile? Because you overwrote the implicit rule with your explicit clean rule.

Why does the clean rule in the second makefile not overwrite the implicit rule? Because it had no recipe. Rules with no recipe do not overwrite the implicit rules, instead they just append prerequisites. See the make manual about multiple rules for more information. See also the make manual about rules with explicit empty recipes.

Why is it an error to not change the working directory before invoking a makefile in a subdirectory? Because make does not change the working directory. Make will work in the inherited working directory. Well technically this is not necessarily an error, but most of the time it is. Do you want the makefiles in the subdirectories to work in the subdirectories? Or do you want them to work in the parent directory?

Why does make ignore the explicit clean rule from the first makefile in the second invocation of clean.sh? Because now the target file clean already exists. Since the rule clean has no prerequisites there is no need to rebuild the target. See the make manual about phony targets which describes exactly this problem.

Why does make search for the target three/makefile in the third invocation? Because make always tries to remake the makefiles before doing anything else. This is especially true if the makefile is explicitly requested using -f but it does not exists. See the make manual about remaking makefiles for more information.

The ifeq() directive has to be in column 1, remove any leading whitespace ie

test_target: 
ifeq ($(TEST),"TRUE")
        echo "Do something"
endif

^ no whitespace