I had no idea that Bash supported so many advanced features! I started working on a simple network program to try and use some of these things I found. My particular program is a basic redis client written completely in Bash. Currently, I can do something like the following:
redis_command -c "ping" redis{001..100}:{6379..6400}
It will send the "ping" command to all of the instances (about 2100) and return the results. I'm currently sending the "ping" command serially, but I thought I could speed up my program by doing all of the initial socket creation in parallel. I tested my program with/without the optimization, and realized that my optimization actually slowed things down. Here's how the optimization looks:
declare -A SOCKETS
function get_socket {
# args: <hostname> <port>
# returns: the socket number, or failure return code
# verify args
if [[ -z "${1}" ]]; then
return 1
fi
if [[ -z "${2}" ]]; then
return 1
fi
local HOSTPORT="${1}:${2}"
if [[ -z ${SOCKETS[${HOSTPORT}]} ]]; then
exec {fd}<>/dev/tcp/${1}/${2}
SOCKETS[${HOSTPORT}]=${fd}
RES=${fd}
else
RES="${SOCKETS[${HOSTPORT}]}"
fi
}
if [[ ${PRECONNECT} -eq 1 ]]; then
echo -n "Pre-connecting to ${#HOSTS[@]} instances... " >&2
for HOST in ${HOSTS[@]}; do
get_socket "${HOST%%:*}" "${HOST##*:}" &
PIDS+=($!)
done
# make sure all of our async connections finished before starting
# TODO, check for errors
for PID in ${PIDS[@]}; do
wait ${PID}
done
echo "done" >&2
fi
Normally, the get_socket() function on its own works great. If I ever need to send a command to a particular server, I call get_socket() ahead of time, and pass the FD to another function that actually sends the command and parses the response. In the pre-connect scenario, I'm calling get_socket in the background via &. Since get_socket()& runs in a separate process, the FD created isn't available/accessible in the calling process, so my optimization is useless.
Is there any way to send sockets in bash, or some other way that I can parallelize my connects without using a separate process?
Best Answer
Not with
/dev/tcp/x/y
directly, but you could use pipes to communicate with processes that are started in the background to connect the TCP sockets and transfer data.Something like
As you end up running two
cat
commands anyway, you might as well do without/dev/tcp
(which is not always enabled) and use for instance socat:Or, you could remove the dependency on bash by using named pipes: