How to use setfacl to give no access to “other” users

You'll notice the "effective" comment that getfacl is throwing out at you. The issue is that permissions are calculating so that "app" isn't getting the write bit set. That's happening because the mask on the file is set to read-only. The mask is used to limit the amount of permissions that could possibly be given out on a particular file or directory.

An example of why you would want this behavior would be like if you knew the file could legitimately need different users/groups to have access to it but for some reason things were getting complicated with permissions and you wanted a way to say "Whatever the other default permissions are set to, whatever their group memberships are, or whatever recursive setfacl gets executed later on, DEFINITELY DON'T GIVE THIS OUT!" The owning user has a special status in the POSIX world, it has rights other users don't have, like the ability to be non-root and change permissions on a file and have its rights not be limited by the mask (which would be pointless anyways because of the first privilege the system gives them). This is why they still get rwx even though the mask is restricted.

To answer your specific question though: add the write bit to the mask on the file and try again as the john user.

here is a command line version of the above explanation, take note of how the "effective" rights change when all I modify is the mask.

Instead of a setuid shell script, consider enabling a specific script with sudo.

Even though it's used most often this way, sudo isn't restricted to "allow someone to execute any program as root". You can easily configure "user A, B and C are allowed to execute only this particular script as root" in /etc/sudoers. See man sudoers for details.

There isn't really an advantage of using sudo instead of a setuid script, except that on systems where setuid scripts are completely disabled for security reasons, the second alternative just won't work at all. You still could write a custom setuid binary, but inserting a line into sudoers is simpler, quicker and easier to change later on when you want to add or remove users.