Shell – Read data from a pipe for a certain amount of time (in seconds)

pipeshellstdout

Is there an easy way to create a pipe that only lasts for a given amount of wall time?

I want to be able to take a "snapshot" of what's being written STDOUT for a period of time (rather than bytes or lines). Something like

tail -f /var/log/whatever.log | pipe_for_seconds 10 > ./10_second_sample_of_log

I'm using BASH, but if this is dramatically easier in some other shell, I'd like to hear about it.

Best Answer

Gnu coreutils since version 7.0 has a timeout command:

timeout 10 tail -f /var/log/whatever.log

If you really need a pipe to timeout for some slightly different procedure, then pipe into timeout:

tail -f /var/log/whatever.log | timeout 10 cat > ./10_second_sample_of_log

Note though that killing some arbitrary part of a pipeline may cause problems due to buffering, depending on signals and program behaviour (this question covers related issues: Turn off buffering in pipe ). It will usually change the exit code of the process too.

If you don't have (a recent) coreutils, this simple timeout program also works well http://www.unixlabplus.com/unix-prog/timeout/timeout.html or the perl approach:

tail -f /var/log/whatever.log | perl -n -e 'BEGIN{alarm 10; $|=1}; print;'

(Note the $|=1 turns off output buffering, this is to prevent loss of output in the pipeline, as referred to above.)

The (slightly ancient) Netpipes package also has a timelimit command (which you can still find on some Linux systems).

This similar question has a few more options: How to introduce timeout for shell scripting?

Related Solutions

Bash Pipe – Using Data Read from a Pipe Instead of a File in Command Options

This is dependent on the command. Some commands that read from a file expect that the file be a regular file, whose size is known in advance, and which can be read from any position and rewinded. This is unlikely if the contents of the file is a list of file names: then the command will probably be content with a pipe which it will just read sequentially from start to finish. There are several ways to feed data via a pipe to a command that expects a file name.

Many commands treat - as a special name, meaning to read from standard input rather than opening a file. This is a convention, not an obligation.
```
ls | command -r -
```
Many unix variants provide special files in /dev that designate the standard descriptors. If /dev/stdin exists, opening it and reading from it is equivalent to reading from standard input; likewise /dev/fd/0 if it exists.
```
ls | command -r /dev/stdin
ls | command -r /dev/fd/0
```
If your shell is ksh, bash or zsh, you can make the shell deal with the business of allocating some file descriptor. The main advantage of this method is that it's not tied to standard input, so you can use standard input for something else, and you can use it more than once.
```
command -r <(ls)
```
If the command expects the name to have a particular form (typically a particular extension), you can try to fool it with a symbolic link.
```
ln -s /dev/fd/0 list.foo
ls | command -r list.foo
```
Or you can use a named pipe.
```
mkfifo list.foo
ls >list.foo &
command -r list.foo
```

Note that generating a list of files with ls is problematic because ls tends to mangle file names when they contain unprintable characters. printf '%s\n' * is more reliable — it'll print every byte literally in file names. File names containing newlines will still cause trouble, but that's unavoidable if the command expects a list of file names separated by newlines.

Pipe the output of parec to sox

The -t option needs to come before the filename it applies to. Also, -t pulse means to read directly from (or write to) the PulseAudio daemon; it's not a file format as such. The type name for raw audio is raw.

Try this:

parec ... | sox -t raw -b 16 -e signed -c 2 -r 44100 - hmm.ogg ...

(where ... means to keep the same arguments you had before)

soxi can't identify the filetype because all it does is look at the header. Raw audio doesn't have a header for it to look at.

Best Answer

Related Solutions

Bash Pipe – Using Data Read from a Pipe Instead of a File in Command Options

Pipe the output of parec to sox

Related Question