Help needed to add text to the end of a string using the “sed” command in Terminal

command lineterminal

In Terminal.app, I used the curl ftp… command to get a directory listing of folders and files from my friend's computer and I directed the output to a file called "curlFTP.txt". Each line of this file appears like these following 3 examples…

  • drwxr-xr-x 1 nobody root 4096 Aug 02 2014 WEDDING_SINGER_4X3
  • drwxr-xr-x 1 nobody root 4096 Aug 02 2014 WEEKEND_AT_BERNIES
  • drwxr-xr-x 1 nobody root 4096 Aug 02 2014 WEIRD_SCIENCE

From each of these lines of that file, I only need to keep the directory names, which would be…

  • WEDDING_SINGER_4X3
  • WEEKEND_AT_BERNIES
  • WEIRD_SCIENCE

Using the cut -c 52- command, I was able to extract only these folder names which I then directed its output to a file I named "curlFTP2.txt"

The next step I needed to do was prepend each line of the file "curlFTP2.txt" with the text…curl ftp://1.1.1.1/Disk\ 1\ Movies/Movies\ 1/ and direct its output to a file named "curlFTP3.txt". Now each line of the file "curlFTP3.txt" would look like these following three examples…

curl ftp://1.1.1.1/Disk 1 Movies/Movies 1/WEDDING_SINGER_4X3
curl ftp://1.1.1.1/Disk 1 Movies/Movies 1/WEEKEND_AT_BERNIES
curl ftp://1.1.1.1/Disk 1 Movies/Movies 1/WEIRD_SCIENCE

Here is the full command I used, which worked incredibly well and was extremely fast, considering there was 760 lines in the file to edit.

cat curlFTP.txt | cut -c 52- > curlFTP2.txt ; sed 's|^|curl ftp://1.1.1.1/Disk\ 1\ Movies/Movies\ 1/|g' curlFTP2.txt > curlFTP3.txt

(the IP address I used in this is a substitute for the IP address I actually used)

Here is where I'm jammed up… what I need to do is use the sed command with the -e argument so I can add an additional sed command to the code from above and add a "/" to the end of each line of that file because each listing is a directory.

I need each line to look like this: curl ftp://1.1.1.1/Disk 1 Movies/Movies 1/WEDDING_SINGER_4X3/
Instead of each line looking like: curl ftp://1.1.1.1/Disk 1 Movies/Movies 1/WEDDING_SINGER_4X3

I was able to figure out that using the sed 's|^|blah|g' command, for example, the "^" character would mean adding "blah" to the beginning of each line.

So here is my question… is there a similar character I can use, like the "^" character, that would allow me to use the sed command to add text to the end of each line?

Best Answer

So here is my question… is there a similar character I can use, like the "^" character, that would allow me to use the sed command to add text to the end of each line?

Yes, it is: $

That said, looking at what is is going on, let me offer you a different over all solution where you do not need to first:

cat curlFTP.txt | cut -c 52- > curlFTP2.txt

You can process the curlFTP.txt file directly and completely with sed:

sed -E -e 's|^.*[A-Za-z]{3}[ ][0-9]{2}[ ][0-9]{4}[ ]|curl ftp://1.1.1.1/Disk 1 Movies/Movies 1/|' -e 's|$|/|' curlFTP.txt > newfile.txt

You could even eliminate > newfile.txtby using the -i option of the sed command:

−i extension
Edit files in-place, saving backups with the specified extension. If a zero-length extension is given, no backup will be saved. It is not recommended to give a zero-length extension when in-place editing files, as you risk corruption or partial content in situations where disk space is exhausted, etc.

sed -i'.bak' -E -e 's|^.*[A-Za-z]{3}[ ][0-9]{2}[ ][0-9]{4}[ ]|curl ftp://1.1.1.1/Disk 1 Movies/Movies 1/|' -e 's|$|/|' curlFTP.txt

Using https://regex101.com to explain the regex used in the sed command:

enter image description here

Which matches e.g.:

drwxr-xr-x 1 nobody root 4096 Aug 02 2014
Related Question