How to make `clear` preserve entire scrollback buffer

Analysis

You call the behavior "weird" but it's by design. clear does not move text out of the screen, it actually clears what's on the screen. Strictly it only tells the terminal (terminal emulator) to do it.

In some terminals clear effectively clears the entire scrollback buffer. At least clear in my Kubuntu does it; with -x I can ask it not to. See man 1 clear.

My $TERM is xterm-256color and this is how clear works for me:

$ clear | od -c
0000000 033   [   H 033   [   2   J 033   [   3   J
0000013
$ clear -x | od -c
0000000 033   [   H 033   [   2   J
0000007

clear -x is equivalent to printf '\033[H\033[2J',
sole clear is equivalent to printf '\033[H\033[2J\033[3J'.

These sequences are ANSI escape codes. The terminal is supposed to interpret them.

Note what clear prints strongly depends on $TERM. E.g. if I pick a "random" terminal:

$ TERM=ztx clear | od -c
0000000 033   E
0000002

The above clear does not use ED; it uses CNL:

which does not clear the screen in my terminal. Maybe it does in a real ztx; or it doesn't, I don't know. Anyway clear "thinks" the terminal is ztx and for ztx CNL is the best choice.

In any case we could build and use an equivalent printf command. The very point of clear is it checks $TERM in the environment and then looks in the terminfo database (see man 5 terminfo) to determine how to clear the screen. With printf we would need to do this manually.

It's not impossible some implementation(s) of clear in some circumstances (i.e. for some value of $TERM) print a sequence that does what you want.

And then there is the terminal (terminal emulator). Even if clear prints \033[2J, the terminal may react by doing what you want. Maybe there is an option that lets you choose. You can write a terminal emulator that reacts this way.

I often use tmux, it implements its own scrollback buffer. Inside it $TERM expands to screen; clear and clear -x are both equivalent to printf '\033[H\033[J'. This sequence does what you want when interpreted by tmux. But outside of tmux (in my case with scrollback buffer provided by Konsole) the same sequence replicates your problem. I deduce tmux is a terminal emulator that reacts the way you want.

My point is: clear prints different things, depending on what it "thinks" your terminal is; then the terminal interprets these things and it may react in whatever way. E.g. your clear seems to do what my clear -x does: it does not clear the entire scrollback buffer, only the visible part. So maybe your clear does not print \033[3J; or maybe it does but your terminal does not clear the buffer fully anyway.

Maybe you could find a different value of $TERM so your clear consulting your terminfo database will be able to make your terminal emulator do what you want. Even if you could do this, I wouldn't say it's the Right Thing in general. Changing $TERM can break other things.

Solution 1: tmux

Use tmux and its scrollback buffer. My tests indicate it behaves the way you want (at least when clear sees $TERM as screen). The tool is a terminal multiplexer, I use it routinely because of its other features and I had to explicitly get out of it to write some parts of this answer.

It may be a major change in how you work in terminals. In my opinion it's worth it. Note using the scrollback buffer of tmux is in many aspects different than using the scrollback buffer of a terminal emulator.

Consider at least trying tmux.

Solution 2: custom clear as a shell function

It seems you don't want to clear anything; you want to move text out of the screen and place the prompt at the top. So let's do this:

clear() (
   if [ "$#" -ne 0 ]; then
      command clear "$@"
      exit
   fi
   h="$(tput lines 2>/dev/null)"
   if [ "$?" -eq 0 ]; then
      until [ "$h" -le 1 ]; do
         printf '\n'
         h=$((h-1))
      done
   fi
   command clear -x
)

The function overrides the clear command when invoked without arguments; it falls back to the command otherwise. When it overrides, the function prints just the right number of newlines to move the previous content out of the screen; then regular clear -x is invoked to clear the newlines and place the prompt at the top.

Notes:

• tput lines is not portable. If there is any problem with this command then the function will only call command clear -x.
• command clear -x assumes your clear executable supports -x like mine does. Judging by the question you don't need -x, your sole clear does not clear the entire buffer (maybe because your clear is different; maybe because your terminal(s) are different). It may be enough to call printf '\033[H\033[2J' instead. On the other hand command clear … consults the terminfo database, it's a feature one may want from the function as well. Adjust the function to your needs.
• If the terminal emulator is resized when the function works then the function may misbehave. If you need to resize, do it before or after calling clear, not during.
• The function can be used inside tmux. It's absolutely not needed there (because tmux itself is a solution) but as far as I can tell it doesn't break anything. So e.g. you can place the function in your ~/.bashrc in case you use Bash outside of tmux, and still the things should work fine when you use Bash inside tmux.
• Any shell function is defined in a shell. Our code can be converted to a script (wrapper) that overrides clear regardless of the shell.