Linux – How does isatty() get the information from the terminal

bashlinux

I noticed that if I type: 

ls --color=auto

or

ls --color=auto | cat
ls --color=auto > >(cat)

I don't see the same output. Therefore I assume a program has an hability to know whether its STDOUT is piped to something. The question is how does it know?

I checked the ENV vars with env and env | cat but I get the same. The answer is somewhere else. I can't figure out where.

From this question I found that I can use isatty() who provide this function? is it part of the shell or the kernel? By going further I see that this function is part of the POSIX standard.

Now I know I have two processes that can communicate in between them with several mechanism:

  • STDIN/STDOUT/STDERR
  • Exit Code
  • Environment variables
  • System calls

Bash and ls are both programs. Thus they can only use the listed mechanisms to exchange information.

The real question behind this is how does isatty() get the information from bash?

Best Answer

The --color option is a feature of the GNU ls program. GNU ls uses the isatty() function to test whether the process's standard output is a TTY. Part of the relevant source code can be seen here:

    case COLOR_OPTION:
      {
        int i;
        if (optarg)
          i = XARGMATCH ("--color", optarg, color_args, color_types);
        else
          /* Using --color with no argument is equivalent to using
             --color=always.  */
          i = color_always;

        print_with_color = (i == color_always
                            || (i == color_if_tty
                                && isatty (STDOUT_FILENO)));

The real question behind this is how does isatty() get the information from bash?

isatty() inspects the file descriptor passed to it to see if the file descriptor represents a TTY (terminal device). The exact way that isatty() works may vary from one system to another. Here is a Darwin implementation from Apple OSX if you're interested:

#include <termios.h>
#include <unistd.h>

int
isatty(fd)
    int fd;
{
    struct termios t;

    return(tcgetattr(fd, &t) != -1);
}

When you run ls --color=auto, your shell (bash) launches the "ls" program using the shell's own standard input, output, and error as the stdin/out/err for the "ls" process. If you are running interactively, then your shell's stdout is probably a terminal, then ls's standard output will probably be a terminal. When ls calls isatty() to test whether its standard output is a terminal, it will probably succeed.

When you run something like ls --color=auto | cat, your shell does three things:

  1. Create a pipe.
  2. Launch cat with its standard input set to be the pipe.
  3. Launch ls with its standard output set to be the pipe.

A pipe is not a terminal. When ls tests whether its standard output is a tty, the test will fail.

Related Question