Is there a way to only execute a command after another is done without a temp file?
I have one longer running command and another command that formats the output and sends it to a HTTP server using curl.
If i just execute commandA | commandB
, commandB
will start curl
, connect to the server and start sending data. Because commandA
takes so long, the HTTP server will timeout.
I can do what I want with commandA > /tmp/file && commandB </tmp/file && rm -f /tmp/file
Out of curiosity I want to know if there is a way to do it without the temp file.
I tried mbuffer -m 20M -q -P 100
but the curl process is still started right at the beginning. Mbuffer waits just until commandA
is done with actually sending the data.
(The data itself is just a few hundred kb at max)
Best Answer
This is similar to a couple of the other answers. If you have the “moreutils” package, you should have the
sponge
command. TryThe
sponge
command is basically a pass-through filter (likecat
) except that it does not start writing the output until it has read the entire input. I.e., it “soaks up” the data, and then releases it when you squeeze it (like a sponge). So, to a certain extent, this is “cheating” – if there’s a non-trivial amount of data,sponge
almost certainly uses a temporary file. But it’s invisible to you; you don’t have to worry about housekeeping things like choosing a unique filename and cleaning up afterwards.The
{ IFS= read -r x; { printf "%s\n" "$x"; cat; } | commandB; }
reads the first line of output fromsponge
. Remember, this doesn’t appear untilcommandA
has finished. Then it fires upcommandB
, writes the first line to the pipe, and invokescat
to read the rest of the output and write it to the pipe.