The default jobs argument for make

There isn't a simple rule that always works. People might recommend a particular figure because they experimented with a particular compilation on a particular machine and this was the best setting, or because they followed some reasoning that may or may not have some relation with reality.

If you're blessed with a lot of RAM, then the limiting factor in a long compilation will be CPU time. Then one task per CPU, plus one pending task for those occasional I/O blocks, is a good setting. That makes it -j3 for a dual-core CPU (or more precisely, for a dual-CPU machine — if each core is hyperthreaded, that would be 4 CPUs, so -j5).

If you have very little RAM, then a limiting factor may be that you can't have many concurrent jobs, or else they'll keep making each other swap out. For example, if you can't comfortably fit two compiler instances in memory, make -j2 may already be slower than make. Since this is dependent on how many compiler processes you can fit in RAM at once, there's no way to derive a general figure.

In between, it may be beneficial to have more jobs. If each compiler process is small, but the build as a whole touches a lot of data, then disk I/O may be the blocking factor. In this case, you'll want several jobs per CPU at once, so that there is always one job using each CPU while others are waiting for I/O. Again, this is very dependent on the build job and on the available RAM, here on what's available for data cache (there's an optimum after which having too many jobs pollutes the cache too much).

Make is simply for convenience. You can build software without it, it is just more difficult.

When you run make to build something, it shows you the commands it is running. You can run those commands manually and get the same effect.

$ echo "int main() {}" > test.c
$ make test
cc     test.c   -o test

Which created a file called test.
I can get the same result by just doing:

cc test.c -o test

If make didn't exist, you could either instruct users to run this cc command by hand, or you could distribute a shell script, and instruct them to run the script.