Clear setuid permission using numeric mode

The short story: it's not possible, no.

The longer story: octal permissions are states. The [+-][rwxXst] notation represents operations that culminate in changing states. Note, not the =[rwxXst] one, which sets state and is equivalent to the octal modes as Gilles said. The X one is the only conditional operation, the other ones are all unconditional.

When you chmod a file with octal permissions, you supply the final state of the permission bits verbatim. When you use the operations, you choose what you want done to the permission bits.

Your question is tantamount to asking if there's a single number that represents all square roots. The answers is obviously ‘no’: every square root yields a number (pedants: though it could be imaginary/irrational), but without the starting state (the number), you can't tell which.

If you want to implement your chgrp -R nobody /whatever while retaining the setuid bit you can use these two find commands

find /whatever ! -type l -perm -04000 -exec chgrp nobody {} + \
                                      -exec chmod u+s {} +
find /whatever ! -type l ! -perm -04000 -exec chgrp nobody {} +

The find ... -perm 04000 option picks up files with the setuid bit set. The first command then applies the chgrp and then a chmod to reinstate the setuid bit that has been knocked off. The second one applies chgrp to all files that do not have a setuid bit.

In any case, you don't want to call chgrp or chmod on symlinks as that would affect their targets instead, hence the ! -type l.