How to identify and remove invisible whitespace characters on gedit

geditwhitespace

I want to remove unnecessary whitespace on my css file. I am using grep with the command as follows:

$ grep -rn "[[:space:]]$"

Surprisingly, it is returning a hit on every line in the file. I search for instances of \t\n, \r\n and \n but could not find anything. How do I go about identifying the invisible whitespace and remove it?

Best Answer

Don't be surprised that your regexps with a \n don't match: The \n is the line separator, it's not in the line. Every line in your file ends with \n-- by definition.* You'll never find a \n inside a line.

One possibility is that you're looking at a Windows file on Unix, and your mystery character is \r (NB not \r\n), which your grep is not recognizing as part of the EOL.

To find out what your lines actually look like, use od -c.

*Footnote for the nitpickers: Except possibly for the final line, and on very old Mac OS systems, etc., etc.

Related Solutions

Ignore whitespaces changes in all git commands

For diff, there's git diff --ignore-space-at-eol, which should be good enough. For diff and blame, you can ignore all whitespace changes with -w: git diff -w, git blame -w.

For git apply and git rebase, the documentation mentions --ignore-whitespace.

For merge, it looks like you need to use an external merge tool. You can use this wrapper script (untested), where favorite-mergetool is your favorite merge tool; run git -c mergetool.nocr.cmd=/path/to/wrapper/script merge. The result of the merge will be in unix format; if you prefer another format, convert everything to that different format, or convert $MERGED after the merge.

#!/bin/sh
set -e
TEMP=$(mktemp)
tr -d '\013' <"$BASE" >"$TEMP"
mv -f "$TEMP" "$BASE"
TEMP=$(mktemp)
tr -d '\013' <"$LOCAL" >"$TEMP"
mv -f "$TEMP" "$LOCAL"
TEMP=$(mktemp)
tr -d '\013' <"$REMOTE" >"$TEMP"
mv -f "$TEMP" "$REMOTE"
favorite-mergetool "$@"

To minimize trouble with mixed line endings, make sure text files are declared as such.

See also Is it possible for git-merge to ignore line-ending differences? on Stack Overflow.

How to scan for invalid characters on gedit

As the file is UTF-8 you could run isutf8. An additional utils package. It gives you both line, char and offset for bad bytes.

Then use xxd, hexdump or the like to analyze.

Unfortunately it stops at first crash. But then again it depends on the file. Could be there is only one bad byte ;)

_{Have some C code that does a similar analysis but for entire file. It is on
a disk somewhere long forgotten. Could try to find it if in need.}

Else yes, the quick and not that dirty way would be to do a diff between a copy saved with gedit – as proposed by the good mr. @vonbrand.

Best Answer

Related Solutions

Ignore whitespaces changes in all git commands

How to scan for invalid characters on gedit

Related Question