Bash – getopts Gets No Arguments When Called Inside While Loop

argumentsbashgetoptsshell-script

I am trying to call a function in a while loop passing some arguments. However, getopts can only get the arguments for the first call.

Here's a minimal example:

function add_all_external_services() {
  env | sed -n "s/^EXTERNAL_SERVICE_OPTIONS_\(.*\)$/\1/p" > options

  while read -r line
  do
    key="${line%%=*}"
    opt="${line#*=}"

    if [[ -n "$key" && -n "$opt" ]]; then
      echo "Adding external service \"$key\" with options: \"$opt\""
      add_external_service $opt
    else
      echo "Missing one or more variables:
  - Key: \"$key\"
  - Options: \"$opt\"
"
    fi
  done < options

  rm options
}

function add_external_service() {
  local local_service_name=""
  local external_service_name=""
  local external_service_namespace=""
  local service_url=""

  echo "    Options: $@"
  while getopts l:s:n:-: OPT; do
    if [[ "$OPT" = "-" ]]; then   # long option: reformulate OPT and OPTARG
      OPT="${OPTARG%%=*}"         # extract long option name
      OPTARG="${OPTARG#$OPT}"     # extract long option argument (may be empty)
      OPTARG="${OPTARG#=}"        # if long option argument, remove assigning `=`
    fi
    case "$OPT" in
      l | local-service-name)           needs_arg; local_service_name="$OPTARG" ;;
      s | external-service-name)        needs_arg; external_service_name="$OPTARG" ;;
      n | external-service-namespace)   needs_arg; external_service_namespace="$OPTARG" ;;
      external-name)                    needs_arg; service_url="$OPTARG" ;;
      ??* )                             die "Illegal option --$OPT" ;;  # bad long option
      \? )                              exit 2 ;;  # bad short option (error reported via getopts)
    esac
  done

  echo "      - local $local_service_name"
  echo "      - name $external_service_name"
  echo "      - namespace $external_service_namespace"
  echo "      - url $service_url"
}

Then, when calling:

export EXTERNAL_SERVICE_OPTIONS_A="-l local_a -s rasa -n botinstance-12424526-review-feature-ce-swdjtf"
export EXTERNAL_SERVICE_OPTIONS_B="-l local_b -s review-feature-ce-swdjtf -n botinstance-12424526-review-feature-ce-swdjtf"

ventury-deploy add_all_external_services

I get this:

Adding external service "B" with options: "-l local_b -s name_b -n namespace_b"
    Options: -l local_b -s name_b -n namespace_b
      - local local_b
      - name name_b
      - namespace namespace_b
      - url 
Adding external service "A" with options: "-l local_a -s name_a -n namespace_a"
    Options: -l local_a -s name_a -n namespace_a
      - local 
      - name 
      - namespace 
      - url

I got the getopts part from here and it works fine whenever I call functions outside loops.

After reading this question I tried adding a & after calling the function inside the loop, and it works… all the args are read by getopts. I can't understand why running the commands in the background would make it work.

If I echo $@ right before the getopts, I can see that all arguments are passed correctly, if I call the second function manually, once for every env variable, it also works correctly.

So, how is running these commands in the background different? I mean, what's different for getopts? Also, why echo $@ can see the arguments while getopts can't?

Best Answer

This is because you are not resetting OPTIND. According to the manual:

Each time it is invoked, getopts places the next option in the shell variable name, initializing name if it does not exist, and the index of the next argument to be processed into the variable OPTIND. OPTIND is initialized to 1 each time the shell or a shell script is invoked.

So OPTIND is used to keep track of the next argument to process and its automatically set to 1 when your script starts but NOT reset when your function ends.

To fix your script just add OPTIND=1 to the start of your function:

function add_external_service() {
  OPTIND=1
  local local_service_name=""
  local external_service_name=""
  local external_service_namespace=""
  local service_url=""

  echo "    Options: $@"
  while getopts l:s:n:-: OPT; do
    if [[ "$OPT" = "-" ]]; then   # long option: reformulate OPT and OPTARG
      OPT="${OPTARG%%=*}"         # extract long option name
      OPTARG="${OPTARG#$OPT}"     # extract long option argument (may be empty)
      OPTARG="${OPTARG#=}"        # if long option argument, remove assigning `=`
    fi
    case "$OPT" in
      l | local-service-name)           needs_arg; local_service_name="$OPTARG" ;;
      s | external-service-name)        needs_arg; external_service_name="$OPTARG" ;;
      n | external-service-namespace)   needs_arg; external_service_namespace="$OPTARG" ;;
      external-name)                    needs_arg; service_url="$OPTARG" ;;
      ??* )                             die "Illegal option --$OPT" ;;  # bad long option
      \? )                              exit 2 ;;  # bad short option (error reported via getopts)
    esac
  done

  echo "      - local $local_service_name"
  echo "      - name $external_service_name"
  echo "      - namespace $external_service_namespace"
  echo "      - url $service_url"
}

Related Solutions

Bash Function – Troubleshoot Strange Behavior in Arguments

Though I can't reproduce the initial run of the function that you have in your question, you should reset OPTIND to 1 in your function to be able to process the function's command line in repeated invocations of it.

From the bash manual:

OPTIND is initialized to 1 each time the shell or a shell script is invoked. When an option requires an argument, getopts places that argument into the variable OPTARG. The shell does not reset OPTIND automatically; it must be manually reset between multiple calls to getopts within the same shell invocation if a new set of parameters is to be used.

From the POSIX standard:

If the application sets OPTIND to the value 1, a new set of parameters can be used: either the current positional parameters or new arg values. Any other attempt to invoke getopts multiple times in a single shell execution environment with parameters (positional parameters or arg operands) that are not the same in all invocations, or with an OPTIND value modified to be a value other than 1, produces unspecified results.

The "shell invocation" that the bash manual mentions is the same as the "single execution environment" that the POSIX text mentions, and both refer to your shell script or interactive shell. Within the script or interactive shell, multiple calls to your lscf will invoke getopts in the same environment, and OPTIND will need to be reset to 1 before each such invocation.

Therefore:

lscf() {
  OPTIND=1

  while getopts f:d: opt ; do
    case $opt in
      f) file="$OPTARG" ;;
      d) days="$OPTARG" ;;
    esac
  done

  echo file is $file
  echo days is $days
}

If the variables file and days should not be set in the calling shell's environment, they should be local variables. Also, quote variable expansions and use printf to output variable data:

lscf() {
  local file
  local days

  OPTIND=1

  while getopts f:d: opt ; do
    case $opt in
      f) file="$OPTARG" ;;
      d) days="$OPTARG" ;;
    esac
  done

  printf 'file is %s\n' "$file"
  printf 'days is %s\n' "$days"
}

Bash – Using getopts for Mandatory Arguments

Maybe something like this?

#!/bin/bash

unset host
unset port
unset user

while getopts h:p:u: opt; do
        case $opt in
                h) host=$OPTARG ;;
                p) port=$OPTARG ;;
                u) user=$OPTARG ;;
                *)
                        echo 'Error in command line parsing' >&2
                        exit 1
        esac
done

shift "$(( OPTIND - 1 ))"

if [ -z "$host" ] || [ -z "$user" ]; then
        echo 'Missing -h or -u' >&2
        exit 1
fi

# The rest of the script continues here and may make use of
# "$host" and "$user", and possibly "$port".

The main bit here is the short if statement at the end that tests whether there were ever anything assigned to the host or user variables in the while loop. If either of these variables are empty, then the code treats it as an error and exits after outputting a short diagnostic message.

The script shown above should also be able to run under /bin/sh as it does not contain any bashishms.

Instead of the if statement, you could use

: ${host:?Missing -h}
: ${user:?Missing -u}

or, even shorter,

: ${host:?Missing -h} ${user:?Missing -u}

The : command is a utility that doesn't do anything, but its argument would still be processed by the calling shell, like for all commands. With the two lines above, the shell would try to expand the host and user variables, and if the variable is unset, or if the expansion results in an empty string, then the string to the right after ? will be outputted and the script will terminate with a non-zero exit status:

$ bash script.sh -h somehost
script.sh: line 21: user: Missing -u
$ echo $?
1

The ${variable:?text} expansion is standard and therefore supported by bash and all other POSIX sh-like shells.

Best Answer

Related Solutions

Bash Function – Troubleshoot Strange Behavior in Arguments

Bash – Using getopts for Mandatory Arguments

Related Question