Bash – understand the output of stty -a and change ctrl backspace to just backspace

If you think that you can cause the event by a specific action or interaction, by far the simplest method is something like:

 watch -d -n1 "stty -F /dev/pts/106 -a | grep -Eo '.(icanon|ixon)'"

Run this on a new terminal, the option to -F is the terminal you will run the program on (run tty to see what it is before starting it). Omit | grep .. if you want to observe the complete terminal state.

Next option, if you are using Linux, is to use ltrace to trace library calls, this similar to strace (and it incorporates some strace capability) but it works with user-space libraries, not just the kernel system calls:

ltrace -tt -e tcgetattr+tcsetattr myprogram ...

This will show and timestamp calls to tcgetattr() and tcsetattr(), the libc functions to get and set terminal attributes.

Ultimately, those libc calls will use the ioctl() system call, this you can trace with strace or truss, here's how to use strace on linux:

strace -tt -e trace=ioctl myprogram [...]

A big advantage here is that strace will happily decode various parameters to syscalls for you.

None of the above will tell you very much about logically where in the program the problem might occur though, to do that you have two options: a debugger, or DLL injection.

Within gdb you can easily set a breakpoint on tcsetattr(), then check the call stack, but this may be tedious if there are many calls (and will require a debug build or symbols for libc to get the best results).

The most comprehensive option (assuming that the target program is dynamically linked) is to inject your own DLL which intercepts or wraps the functions you need to track, in this case tcsetattr().

#define _GNU_SOURCE
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <dlfcn.h>
#include <unistd.h>
#include <termios.h>

/* 
 * gcc -nostartfiles -shared -ldl -Wl,-soname,tcsetattr -o tc.so wraptc.c
 * LD_PRELOAD=./tc.so stty -icanon ixon
 *
 */

#define DEBUG 1
#define dfprintf(fmt, ...) \
    do { if (DEBUG) fprintf(stderr, "[%14s#%04d:%8s()] " fmt, \
          __FILE__, __LINE__, __func__, __VA_ARGS__); } while (0)


typedef int tcsetattr_fp(int fd, int optional_actions, 
                     const struct termios *termios_p);
static tcsetattr_fp    *real_tcsetattr;

void _init()
{
    dfprintf("It's alive!\n","");
    real_tcsetattr = dlsym(RTLD_NEXT, "tcsetattr");
    dfprintf("Hooked %p tcsetattr()\n",(void *)real_tcsetattr);
}

int tcsetattr(int fd, int optional_actions, const struct termios *termios_p)
{
     void *bt[20];
     size_t btsz;

     int rc,stacktr=0;
     dfprintf("Caught tcsetattr(%i,%04x,...)\n",fd,optional_actions);

     if ( (fd==0) && !((termios_p->c_lflag) & ICANON)) {
         dfprintf("ICANON off!\n","");
         stacktr=1;
     }
     if ( (fd==0) && !((termios_p->c_iflag) & IXON)) {
         dfprintf("IXON off!\n","");
         stacktr=1;
     }
     if (stacktr) {
         btsz=backtrace(bt,sizeof(bt));
         backtrace_symbols_fd(bt,btsz,STDERR_FILENO);
     }

     rc=real_tcsetattr(fd,optional_actions, termios_p);
     return rc;
}

Compile and invoke as indicated in the comments. This code locates the real libc tcsetattr() function, and contains an alternative version which is used instead. This code calls backtrace() when it sees possibly interesting activity on FD 0, then calls the real libc version. It may require minor adjustment.

The first step in understanding what's going on is to be aware that there are in fact two “newline” characters. There's carriage return (CR, Ctrl+M) and line feed (LF, Ctrl+J). On a teletype, CR moves the printer head to the beginning of the line while LF moves the paper down by one line. For user input, there's only one relevant concept, which is “the user has finished entering a line”, but unfortunately there's been some divergence: Unix systems, as well as the very popular C language, use line feed to represent line breaks; but terminals send a carriage return when the user presses the Return or Enter key.

The icrnl setting tells the terminal driver in the kernel to convert the CR character to LF on input. This way, applications only need to worry about one newline character; the same newline character that ends lines in files also ends lines of user input on the terminal, so the application doesn't need to have a special case for that.

By default, ghci, or rather the haskeline library that it uses, has a key binding for Ctrl+J, i.e. LF, to stop accumulating input and start processing it. It has no binding for Ctrl+M i.e. CR. So if the terminal isn't converting CR to LF, ghci doesn't know what to do with that character.

Haskeline instructs the terminal to report keypad keys with escape sequences. It queries the terminal's terminfo settings to know what those escape sequences are (kent entry in the terminfo database). (The terminfo database is also how it knows how to enable keypad escapes: it sends the smkx escape sequence, and it sends rmkx on exit to restore the default keypad character mode.) When you press the Enter key on the keypad in ghci, that sends the escape sequence \eOM, which haskeline recognizes as a binding to stop accumulating input and start processing it.