How to partially serialize with GNU make

gnugnu-makemakesunpro

When I have a make task where a specific target needs to be made before another, while in parallel mode, this is simple when using SunPro Make (dmake). The following makefile:

install:  dir dir/file

dir:
    mkdir dir

dir/file:
    cp source dir/file

could be made parallel make safe by changing the first line to:

install: dir .WAIT dir/file

or by using:

.NO_PARALLEL: install

The rest of the makefile will still be handled in parallel mode and even a list if targets to the left or to the right of .WAIT will be handled in parallel mode.

See: http://schilytools.sourceforge.net/man/man1/make.1s.html and http://schilytools.sourceforge.net/man/man1/dmake.1.html

but GNU make does not seem to have a similar option. Is there really no way to do this with GNU make?

To be more specific, the solution needs to be written in a way that allows the makefile to be processed with other make implementations.

The special target .WAIT could e.g. be in a make program dependent macro that is called e.g. $(WAIT)

Note that there is an apparent solution that does not help:

In theory one could make dir a dependency of dir/file in order to enforce to create dir before trying to copy to dir/file, but if you later copy another file into dir, this makes dir newer than dir/file. As a result, calling make more than once would copy to dir/fileagain and again, without the expected result from make to do the copy only in case that the source has become newer that dir/file.

This raises the alternate question whether there may be a topology of dependencies that enforces make to create dir before copying to dir/file without making dira dependency of dir/file.

Best Answer

In GNU make, you can use order-only-prerequisites to make sure dir is created before the rule to create dir/file is run.

See this page for more details.

You can see from the example in GNU make's manual that it's really tailored for adding dependency on a directory being created before creating files under it.

In the case of your specific example, something like this should do it:

dir/file: | dir
        cp source dir/file

Though you don't need to modify the existing rule, you could just add that dependency somewhere else in your Makefile...

To be more specific, the solution needs to be written in a way that allows the makefile to be processed with other make implementations.

I don't think this would work in other make implementations... One possible solution is to create a GNUmakefile that would include the actual Makefile and also add GNU make specific extensions.

You could also probably make GNU make ignore a rule such as .WAIT by defining it as a dummy rule in your GNUmakefile.

I hope these pointers are helpful!

Related Solutions

Make – Strange Behavior Automating with Makefile

That behaviour is not a bug. It is a feature. A feature and a possible user error to be precise.

The feature in question is one of the implicit rules of Make. In your case the implicit rule to "build" *.sh files. The user error, your error, is not changing the working directory before invoking the makefile in the subdirectories.

TL; DR: to fix this you can do one or more of the following:

Fix the shell script to change the working directory:

#!/bin/bash

for f in *; do
        if [[ -d $f && -f $f/makefile ]]; then
                echo "Making clean in $f..."
                (cd $f; make clean)
        fi
done

Make the empty rules explicit:
```
clean: ;
```
Make the clean targets phony:
```
.PHONY: clean
```

Detailed explanation:

Make has a bunch of implicit rules. This allows one to invoke make on simple projects without even writing a makefile.

Try this for a demonstration:

create an empty directory and change in to the directory
create a file named clean.sh.
run make clean

Output:

$ make clean
cat clean.sh >clean 
chmod a+x clean

BAM! That is the power of implict rules of make. See the make manual about implicit rules for more information.

I will try to answer the remaining open question:

Why does it not invoke the implicit rule for the first makefile? Because you overwrote the implicit rule with your explicit clean rule.

Why does the clean rule in the second makefile not overwrite the implicit rule? Because it had no recipe. Rules with no recipe do not overwrite the implicit rules, instead they just append prerequisites. See the make manual about multiple rules for more information. See also the make manual about rules with explicit empty recipes.

Why is it an error to not change the working directory before invoking a makefile in a subdirectory? Because make does not change the working directory. Make will work in the inherited working directory. Well technically this is not necessarily an error, but most of the time it is. Do you want the makefiles in the subdirectories to work in the subdirectories? Or do you want them to work in the parent directory?

Why does make ignore the explicit clean rule from the first makefile in the second invocation of clean.sh? Because now the target file clean already exists. Since the rule clean has no prerequisites there is no need to rebuild the target. See the make manual about phony targets which describes exactly this problem.

Why does make search for the target three/makefile in the third invocation? Because make always tries to remake the makefiles before doing anything else. This is especially true if the makefile is explicitly requested using -f but it does not exists. See the make manual about remaking makefiles for more information.

Restricting GNU‑Make to POSIX Make behaviour

I'm pretty sure that there is no way to disable GNU make extensions. The .POSIX target improves the compatibility but does not disable compatible extensions such as functions, proprietary dot targets, % patterns in targets, etc.

I don't know of any make implementation that sticks to POSIX features. For example MINIX, Solaris, OpenBSD all have their own extensions. You could at least try to build your programs with these four implementations (assuming that your programs do build on all these platforms), which will give you a pretty good portability assurance in practice. Add Windows with the make utility from Visual Studio, plus a port of some Unix shell tools, if you can use a non-free OS (I think there's a free version of VS that would be enough).

Best Answer

Related Solutions

Make – Strange Behavior Automating with Makefile

Restricting GNU‑Make to POSIX Make behaviour

Related Question