Command Redirection – How to Redirect stdin and stdout to Output

pipestdinstdouttee

Suppose I have the Python script:

#!/usr/bin/env python
input('Y/n: ')
print('Next line')

After I press Y, I want both the terminal and my output.txt containing:

Y/n: Y
Next line

Running the following I don't get Y in the output:

$ python ask.py 2>&1 | tee output.txt

How can I include my response in the output?

Best Answer

Along the lines of what you already have (tested with python3):

tee -a output.txt | python ask.py 2>&1 | tee -a output.txt

As a downside, you will need to explicitly type something after the python script terminates, to make the first tee attempt writing to the pipe, receive a SIGPIPE and exit. You may be able to overcome this limitation with:

tee -a output.txt | { python ask.py 2>&1; kill 0; } | tee -a output.txt

Where kill is used to kill all the processes in the current process group (i.e. the dedicated process group the pipeline is run as). (But note that caveats may apply).

An alternative, if script is available to you, may be:

script -q -c 'python ask.py' output.txt

In this case python will be connected to a pseudo-terminal device, ensuring it behaves as if it were run interactively on a terminal.

Related Solutions

Bash – How to Use tee to Redirect to grep

$ ps aux | tee >(head -n1) | grep syslog
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND 
syslog     806  0.0  0.0  34600   824 ?        Sl   Sep07   0:00 rsyslogd -c4

The grep and head commands start at about the same time, and both receive the same input data at their own leisure, but generally, as data becomes available. There are some things that can introduce the 'unsynchronized' output which flips lines; for example:

The multiplexed data from tee actually gets sent to one process before the other, depending primarily on the implementation of tee. A simple tee implementation will read some amount of input, and then write it twice: Once to stdout and once to its argument. This means that one of those destinations will get the data first.

However, pipes are all buffered. It is likely that these buffers are 1 line each, but they might be larger, which can cause one of the receiving commands to see everything it needs for output (ie. the grepped line) before the other command (head) has received any data at all.
Notwithstanding the above, it's also possible that one of these commands receives the data but is unable to do anything with it in time, and then the other command receives more data and processes it quickly.

For example, even if head and grep are sent the data one line at a time, if head doesn't know how to deal with it (or gets delayed by kernel scheduling), grep can show its results before head even gets a chance to. To demonstrate, try adding a delay: ps aux | tee >(sleep 1; head -n1) | grep syslog This will almost certainly output the grep output first.

$ ps aux | tee >(grep syslog) | head -n1
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

I believe you often only get one line here, because head receives the first line of input and then closes its stdin and exits. When tee sees that its stdout has been closed, it then closes its own stdin (output from ps) and exits. This could be implementation-dependent.

Effectively, the only data that ps gets to send is the first line (definitely, because head is controlling this), and maybe some other lines before head & tee close their stdin descriptors.

The inconsistency with whether the second line appears is introduced by timing: head closes stdin, but ps is still sending data. These two events are not well-synchronized, so the line containing syslog still has a chance of making it to tee's argument (the grep command). This is similar to the explanations above.

You can avoid this problem altogether by using commands that wait for all input before closing stdin/exiting. For example, use awk instead of head, which will read and process all its lines (even if they cause no output):

ps aux | tee >(grep syslog) | awk 'NR == 1'

But note that the lines can still appear out-of-order, as above, which can be demonstrated by:

ps aux | tee >(grep syslog) | (sleep 1; awk 'NR == 1')

Hope this wasn't too much detail, but there are a lot of simultaneous things interacting with each other. Separate processes run simultaneously without any synchronization, so their actions on any particular run can vary; sometimes it helps to dig deep into the underlying processes to explain why.

Command Completion Output – How to Redirect

You could make use of the builtin compgen:

compgen: compgen [-abcdefgjksuv] [-o option]  [-A action] [-G globpat]
[-W wordlist]  [-F function] [-C command] [-X filterpat] [-P prefix]
[-S suffix] [word]

    Display possible completions depending on the options.

    Intended to be used from within a shell function generating possible
    completions.  If the optional WORD argument is supplied, matches against
    WORD are generated.

    Exit Status:
    Returns success unless an invalid option is supplied or an error occurs.

TAB at the prompt would list commands, shell builtins, keywords, aliases and functions. So you could say:

compgen -cbka -A function | grep '^y' > myfile.txt

to get all the options that you see upon typing yTAB at the shell prompt into the file myfile.txt.

Eliminate the grep pipeline in order to get all the possible commands, functions, ... into the file:

compgen -cbka -A function > myfile.txt

Best Answer

Related Solutions

Bash – How to Use tee to Redirect to grep

Command Completion Output – How to Redirect

Related Question