How to use pv to show progress of openssl encryption / decryption

encryptionopensslpv

I need to encrypt and be able to decrypt files with openssl, currently I do this simply with:

openssl enc -aes-256-cbc -salt -in "$input_filename" -out "$output_filename"

and the decryption with:

openssl enc -aes-256-cbc -d -salt -in "$input_filename" -out "$output_filename"

But with large files, I would like to see progress.

I tried different variations of the following (decryption):

pv "$input_filename" | openssl enc -aes-256-cbc -d -salt | pv > "$output_filename"

But this fails to ask me for a password. I am unsure as to how to go about it?

EDIT1:

I found this tar over openssl:

https://stackoverflow.com/a/24704457/1997354

While it could be extremely helpful, I don't get it much.

EDIT2:

Regarding the named pipe:

It almost works. Except for the blinking progress, which I can't show you obviously and the final result looking like this:

enter aes-256-cbc decryption password:
1.25GiB 0:00:16 [75.9MiB/s] [==============================================================================================================================================================================================>] 100%            
1.25GiB 0:00:10 [ 126MiB/s] [                                             <=>                                                                                                                                                                ]

Best Answer

You should try

openssl enc -aes-256-cbc -d -salt -in "$input_filename" | pv -W >> "$output_filename"

From the Manual:

-W, --wait:

Wait until the first byte has been transferred before showing any progress information or calculating any ETAs. Useful if the program you are piping to or from requires extra information before it starts, eg piping data into gpg(1) or mcrypt(1) which require a passphrase before data can be processed.

which is exactly your case. If you need to see the progress bar, for the reason clearly explained by Weijun Zhou in a comment below, you can reverse the order of the commands in the pipe:

pv -W "$input_filename" | openssl enc -aes-256-cbc -d -salt -out "$output_filename"

Related Solutions

How to show progress with GPG for large files

progress can do this for you — not quite a progress bar, but it will show progress (as a percentage) and the current file being processed (when multiple files are processed):

gpg ... &
progress -mp $!

Bash Scripting – How to Grep for Unicode in a Bash Script

grep is the wrong tool for the job.

You see the � U+FFFD REPLACEMENT CHARACTER not because it’s literally in the file content, but because you looked at a binary file with a tool that is supposed to handle only text-based input. The standard way to handle invalid input (i.e., random binary data) is to replace everything that is not valid in the current locale (most probably UTF-8) with U+FFFD before it hits the screen.

That means it is very likely that a literal \xEF\xBF\xBD (the UTF-8 byte sequence for the U+FFFD character) never occurs in the file. grep is completely right in telling you, there is none.

One way to detect whether a file contains some unknown binary is with the file(1) command:

$ head -c 100 /dev/urandom > rubbish.bin
$ file rubbish.bin
rubbish.bin: data

For any unknown file type it will simply say data. Try

$ file out.txt | grep '^out.txt: data$'

to check whether the file really contains any arbitrary binary and thus most likely rubbish.

If you want to make sure that out.txt is a UTF-8 encoded text file only, you can alternatively use iconv:

$ iconv -f utf-8 -t utf-16 out.txt >/dev/null

Best Answer

Related Solutions

How to show progress with GPG for large files

Bash Scripting – How to Grep for Unicode in a Bash Script

Related Question