What does the “s” mean in the following permission:“-rwsr-x—”

chmod might or might not change the permissions of files that are already set to what you want, but if not, it would still need to check them to see what their current permissions are[0]. With hundreds of thousands of files, I don't think it would matter either way; the time is most likely being spent by the tools stating every file.

You can try using find to either check for files newer than the last run or files that need chmod to be run, but I don't think you'll get much speed improvement.

If possible for your script, you might be able to get the new files put into a separate directory first, as a "holding" area. Then you can chmod THAT directory (which only has new files), and mv them in with the rest. That should be substantially faster, but unfortunately won't work for every application.

[0] Even if it does try to set the permission of files that don't need any changes, the underlying filesystem probably won't do anything with the request, because it's unnecessary.

The 050 should be clear, that sets read and execute bits for the group the first 6 sets the set-user-ID and set-group-ID bits (see man 2 chmod). Effectively this means that executing container-extractor can only be done by root or members of the group hadoop and that the executable runs with effective uid being root and effective gid being hadoop.