Bash – Convert password with special characters for use with expect script

bashexpectregular expressionsftp

I have an expect script that connects to an SFTP site and uploads some files. The ID and password are contained in local text files. The expect script reads those files in as a variable and then passes it to the SFTP server.

The problem I am experiencing is we are being provided passwords with complexity that includes special characters ($\,!) etc. For example if the password is pass$word the $ passes as a special rather than a literal. Being this is being passed through a variable (multiple sites and IDs)

I am unable to use '$pw' as it will literally pass $pw to the server and using "$pw" sends special characters. I need it to pass the password exactly as is.

I have been escaping them out by hand so far (e.g. pass\$word) but that is tedious and I would like to do it script-o-matically.

The script looks like this (names and places changed to protect the innocent)

#!/bin/bash

home=/data/ftp/vf/drop/
un=`cat /home/vf/workflow/.udf/.u/.u$1`
pw=`cat /home/vf/workflow/.udf/.p/.p$1`
sl=`cd /home/vf/workflow/scheduler; grep $1 upload*|cut -d \: -f1`

/usr/bin/expect -c "
spawn /usr/bin/sftp -o KexDHMin=1024 $un@$sl.mysteriouslocation.com
set timeout -1
expect *Authentication
expect Password*
send \"$pw\r\"
expect sftp*
send \"mput /$home/$1/transfer/*\r\"
expect sftp*
send \"ls \r\"
expect sftp*
send \"bye\r\"
expect eof
exit

How can I either pass the password to the expect script so that it sends the literal characters and not give them special meaning?
I don't think I can script a "fix" for the password file itself, meaning go in and every time it sees a special character, escape it out via sed or other means as the escape character itself is special and could end up in a loop.

Your help is greatly appreciated!

Best Answer

A small Perl snippet can be used to escape (backslash) all ASCII characters not matching /[A-Za-z_0-9]/ by using the quotemeta function. The Perl command line is easily incorporated into your shell script:

#/bin/sh
pstr='$x%y!zpass' # plain, un-escaped string
estr=$(perl -e 'print quotemeta shift(@ARGV)' "${pstr}")
echo ${estr}      # show escaped string

produces:

\$x\%y\!zpass

Related Solutions

Why do some regex commands have opposite intepretations of ‘\’ with various characters

The answer is really "just because". There's a whole bunch of different regular expression syntaxes, and while they share a similar appearance and usually the basics are the same, they vary in the particulars.

Historically, every tool had its own new implementation, doing whatever the author thought best. There's a balance between making characters special with and without escaping — too many characters that are "naturally special" and you end up having to escape them all the time just to match on them; or, the other way around, you end up needing a bunch of escapes to use common regex syntax like () grouping. And everyone writing a program decided how to do it based on the needs of what their program matched against, on what they felt was the right approach, and on the phase of the moon.

There's an attempt at standardization from POSIX, which defines "basic regular expressions" and "extended regular expressions". Awesomely, these work backwards from each other in regards to \ — sometimes, but not with perfect consistency.

Perl regular expressions have become another defacto standard, for two reasons: first, they're very flexible and powerful, and second, they're actually pretty sane, with conventions like "\ always escapes a non-alphanumeric character".

GNU Find has a -regextype option, where you can change the regular expression syntax used. Sadly, "perl" is not an option, at least in the version of find I have. (The default is, not surprisingly from GNU, "emacs", and that syntax is documented here.)

Ubuntu – How to use `expect` with `zipcloak` II

I'd write that zipcloak-er as

#!/usr/bin/env expect
set zipfile [lindex $argv 0]
if {![file exists $zipfile]} {error "no such zipfile: $zipfile"}
set pass "mypassword"
spawn zipcloak $zipfile
expect {
    -re {password: $} {send "$pass\r"; exp_continue}
    eof
}

Since the "enter password: " and "verify password: " prompts have a common suffix, it makes sense to use exp_continue to reuse the code.

"does not work" is not helpful. Did you use exp_internal 1? Try this:

#!/bin/bash
MYPWD="mypassword"

expect -c " 
    exp_internal 1
    spawn zipcloak test.upd
    expect {
        -re {password: \$} {send \"$MYPWD\\r\"; exp_continue}
        eof
    }
"

Sorry Alex, I don't understand your issue. This is what happens for me in bash:

$ zip test.upd file
  adding: file (deflated 22%)
$ MYPWD="mypassword"
$ expect -c " 
    exp_internal 1
    spawn zipcloak test.upd
    expect {
        -re {password: \$} {send \"$MYPWD\\r\"; exp_continue}
        eof
    }
"

spawn zipcloak test.upd
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {27485}
Gate keeper glob pattern for 'password: $' is 'password: '. Activating booster.

expect: does "" (spawn_id exp4) match regular expression "password: $"? Gate "password: "? gate=no
Enter password: 
expect: does "Enter password: " (spawn_id exp4) match regular expression "password: $"? Gate "password: "? gate=yes re=yes
expect: set expect_out(0,string) "password: "
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) "Enter password: "
send: sending "mypassword\r" to { exp4 }
expect: continuing expect

expect: does "" (spawn_id exp4) match regular expression "password: $"? Gate "password: "? gate=no

Verify password: 
expect: does "\r\nVerify password: " (spawn_id exp4) match regular expression "password: $"? Gate "password: "? gate=yes re=yes
expect: set expect_out(0,string) "password: "
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) "\r\nVerify password: "
send: sending "mypassword\r" to { exp4 }
expect: continuing expect

expect: does "" (spawn_id exp4) match regular expression "password: $"? Gate "password: "? gate=no


expect: does "\r\n" (spawn_id exp4) match regular expression "password: $"? Gate "password: "? gate=no
encrypting: file

expect: does "\r\nencrypting: file\r\n" (spawn_id exp4) match regular expression "password: $"? Gate "password: "? gate=no
expect: read eof
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) "\r\nencrypting: file\r\n"
argv[0] = expect  argv[1] = -c  argv[2] =  
    exp_internal 1
    spawn zipcloak test.upd
    expect {
        -re {password: $} {send "mypassword\r"; exp_continue}
        eof
    }

set argc 0
set argv0 "expect"
set argv ""

Best Answer

Related Solutions

Why do some regex commands have opposite intepretations of ‘\’ with various characters

Ubuntu – How to use `expect` with `zipcloak` II

Related Question