Linux – Batch renaming of files in a directory using sed

bashlinuxsedUbuntu

I run a script which generated about 10k files in a directory. I just discovered that there is a bug in the script which causes some filenames to have a carriage return (presumably a '\n' character).

I want to run a sed command to remove the carriage return from the filenames.

Anyone knows which params to pass to sed to clean up the filenames in the manner described?

I am running on Linux (Ubuntu)

UPDATE

The character causing the filename to 'break up' accross multiple lines appear to be a CR (carriage return) instead of '\n'. The filename is being diaplayed in thetitle of a text editor with %0D in the positions of where the file name breaks up. So I need to remove the CR chars from my filenames.

Best Answer

Try with rename:

find /path/to/dir -exec rename -n 's/\n/_/' {} \+

Remove -n to actually rename the files.

Related Solutions

Linux – Using sed to enable .htaccess files in Apache config

sed supports pattern ranges in this form:

sed '/startpattern/,/endpattern/ <sed-commands>' file

For example:

sed '/<Directory \/var\/www\/>/,/<\/Directory>/ s/AllowOverride None/AllowOverride all/' httpd.conf

Fill the last line up with nothing but emptiness to a specific position

Try this:

sed -i -e 's/.\{80\}/&\r\n/g' \
  -e 's/$/################################################################################/' \
  -e 's/\(\r\n.\{80\}\)#*$/\1/' "C:\Folder\test.txt"

(I have broken the command into multiple lines just for readability in presentation; you can do it all on one line.) The s/$/###…###/ appends 80 # characters, and then the s/$\r\n.\{80\}$#*$/\1/ looks for this sequence:

\r\n
80 of any character
any number of #s
up to the end of the buffer

and replaces it with the group formed by the first two bullets. In other words, it keeps the n characters that are the last line of the file and the 80-n # characters that follow it, and discards the n # characters that follow those.

Once you've verified that this works, just change all occurrences of # in the above command to spaces.

OP's Final Working Solution

^{NOTE: The SET s10= line has 10 empty spaces in it to the right of the equal sign.}

Batch Script

@ECHO ON

SET File=C:\Folder\test.txt
::: The below s10 variable equals ten white\empty spaces - so it has 10 blank spaces in it
SET s10=          
SET s40=%s10%%s10%%s10%%s10%
SET s80=%s40%%s40%

sed -i -e "s/.\{80\}/&\r\n/g" -e "s/$/%s80%/" -e "s/\(\r\n.\{80\}\)\s*$/\1/" "%File%"
GOTO :EOF

We can make this a little easier to read and less tedious to type.

Pure sed

If your text is all alphanumeric, and there is some character (such as @) that will never ever appear in your data, then you can do something like this:
```
  -e 's/$/@@@@@@@@@@/' \        # Append 10 @ characters to the line.
  -e 's/@/        /g' \         # Change each @ to eight spaces.
```
If you want to make your script unambiguously readable, at the cost of making it somewhat more verbose, take it one step further:
```
  -e 's/$/@@@@@@@@@@/' \        # Append 10 @ characters to the line.
  -e 's/@/@@@@@@@@/g' \         # Change each @ to eight @s (there are now 80 @s).
  -e 's/@/ /g'        \         # Change each of the 80 @s to a space.
```
If there's no single character that never appears in your data, but there is some (short) character sequence, like (), that never appears, you can use the same principle:
```
  -e 's/$/()()()()()()()()()()/' \  # Append 10 () pairs to the line.
  -e 's/()/        /g' \            # Change each () to eight spaces.
```
Be careful when dealing with characters that are special in regular expressions; e.g., ., *, [, ^, &, etc. (And don't include the #-comments in your script in those places; to break a simple command across multiple lines, the \ must be the last character on the line (before the newline) and not in a comment.)

Using the shell

Same principle:

#!/bin/sh
      ︙
s="          "                  # Set $s to 10 spaces.
s80="$s$s$s$s$s$s$s$s"          # Set $s80 to 8 copies of $s; i.e., 80 spaces.
      ︙
sed -i -e 's/.\{80\}/&\r\n/g' \
  -e 's/$/'"$s80"'/' \
  -e 's/\(\r\n.\{80\}\)\s*$/\1/' "C:\Folder\test.txt"