Bash – In bash, can I asynchronously open sockets and use them later

bashnetworkingsocket

I had no idea that Bash supported so many advanced features! I started working on a simple network program to try and use some of these things I found. My particular program is a basic redis client written completely in Bash. Currently, I can do something like the following:

redis_command -c "ping" redis{001..100}:{6379..6400}

It will send the "ping" command to all of the instances (about 2100) and return the results. I'm currently sending the "ping" command serially, but I thought I could speed up my program by doing all of the initial socket creation in parallel. I tested my program with/without the optimization, and realized that my optimization actually slowed things down. Here's how the optimization looks:

declare -A SOCKETS
function get_socket {
  # args: <hostname> <port>
  # returns: the socket number, or failure return code

  # verify args
  if [[ -z "${1}" ]]; then
    return 1
  fi
  if [[ -z "${2}" ]]; then
    return 1
  fi

  local HOSTPORT="${1}:${2}"
  if [[ -z ${SOCKETS[${HOSTPORT}]} ]]; then
    exec {fd}<>/dev/tcp/${1}/${2}
    SOCKETS[${HOSTPORT}]=${fd}
    RES=${fd}
  else
    RES="${SOCKETS[${HOSTPORT}]}"
  fi
}

if [[ ${PRECONNECT} -eq 1 ]]; then
  echo -n "Pre-connecting to ${#HOSTS[@]} instances... " >&2
  for HOST in ${HOSTS[@]}; do
    get_socket "${HOST%%:*}" "${HOST##*:}" &
    PIDS+=($!)
  done

  # make sure all of our async connections finished before starting
  # TODO, check for errors
  for PID in ${PIDS[@]}; do
    wait ${PID}
  done
  echo "done" >&2
fi

Normally, the get_socket() function on its own works great. If I ever need to send a command to a particular server, I call get_socket() ahead of time, and pass the FD to another function that actually sends the command and parses the response. In the pre-connect scenario, I'm calling get_socket in the background via &. Since get_socket()& runs in a separate process, the FD created isn't available/accessible in the calling process, so my optimization is useless.

Is there any way to send sockets in bash, or some other way that I can parallelize my connects without using a separate process?

Best Answer

Not with /dev/tcp/x/y directly, but you could use pipes to communicate with processes that are started in the background to connect the TCP sockets and transfer data.

Something like

coproc hostA {
  exec {fd}<> /dev/tcp/127.0.0.1/80
  cat <&$fd & cat >&$fd
}

# same for hostB...

echo -e 'HEAD / HTTP/1.0\r\n\r' >&${hostA[1]}
cat <&${hostA[0]}

As you end up running two cat commands anyway, you might as well do without /dev/tcp (which is not always enabled) and use for instance socat:

coproc hostA { socat - tcp:localhost:80; }

Or, you could remove the dependency on bash by using named pipes:

mkfifo hostA.in hostA.out
socat - tcp:localhost:80 < hostA.in > hostA.out &
Related Question