Terminal paste unreliable

copy/pasteterminal

I'm trying to paste text into Terminal (v2.5 build 343, I'm running Yosemite), and it seems to consistently omit characters.

For example if I copy the following text to the pasteboard:

1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890

Then in Finder Edit -> Show Clipboard I can verify that the text is all there.

Next I open Terminal and type:

cat > test

Then paste the above text and press Ctrl+d.

On the screen I get this:

1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
1234567890,12345678,1234567890,1234567890,1234567890
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890

but the contents of the file "test" match the original text which I copied to the pasteboard.

Is there some reason why Terminal would drop characters when echoing to the screen during a paste operation?

Best Answer

Stdout in this case is line-buffered and is 1024 bytes which is causing the output truncation on the display. When you paste from the clipboard the non-printable New-Line (EOL) characters at the end of each line are simply counted as another character on the line as far as the buffer count is concerned. The reason the math doesn’t add up for where the truncation happens that @miken32 was trying to calculate, is because you are not counting the New-Line characters.

If you cat the test file using a –e option you can see the non printable EOL characters as $, including them in the math should add up.

$ cat -e test
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$

You can get around filling the line buffer with pasted bytes if you specify your own end-of-file (EOF) marker to the cat command. This way when the cat command encounters the EOL characters it will reset the line buffer at the beginning of each new line instead of just counting them as another byte in a continuous stream because the cat command is parsing each new line looking for the EOF string on a line by itself to know when to exit.

You can do something like this :

cat << EOF > test

The stdin redirect (<< EOF) tells cat to keep printing until it encounters input matching the specified end-of-file characters on a new line by themselves. This way stdout will then print every pasted printable character line by line, the drawback is that you get a “>" character in the output at the start of each new line.

$ cat << EOF > test
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> EOF

The end-of-file marker is typed literally as the characters E, O, F, after your paste and can be anything you like. Also the EOF characters, and the > characters are not included in the redirected output sent to the file. Typing the letters as EOF is just symbolic and can be anything you like, XXX for example, you just have to be certain that what ever EOF marker you specify does not appear in the paste buffer.

For the record, you can still use ^D as the EOF marker when using stdin, even though you specify something else. The use of the EOF string is an old school scripting convention for delimiting a block of text to redirect from within the script.

Hope this helps.