If I mount a simple loop device,
losetup -a
give me the devices opened.
Is something similar possible with cryptsetup
?
cryptsetupluks
If I mount a simple loop device,
losetup -a
give me the devices opened.
Is something similar possible with cryptsetup
?
It doesn't seem to be possible with the cryptsetup
command. Unfortunately cryptsetup
has a few such immutable flags... --allow-discards
is also one of them. If this wasn't set at the time you opened the container, you can't add it later.
At least, not with the cryptsetup
command. However, since cryptsetup
creates regular Device Mapper targets, you can resort to dmsetup
to modify them. Of course, this isn't recommended for various reasons: it's like changing the partition table of partitions that are in use - mess it up and you might lose all your data.
The device mapper allows dynamic remapping of all devices at runtime and it doesn't care about the safety of your data at all; which is why this feature is usually wrapped behind the LVM layer which keeps the necessary metadata around to make it safe.
Create a read-only LUKS device:
# truncate -s 100M foobar.img
# cryptsetup luksFormat foobar.img
# cryptsetup luksOpen --read-only foobar.img foobar
The way dmsetup
sees it:
# dmsetup info foobar
Name: foobar
State: ACTIVE (READ-ONLY)
Read Ahead: 256
Tables present: LIVE
[...]
# dmsetup table --showkeys foobar
0 200704 crypt aes-xts-plain64 ef434503c1874d65d33b1c23a088bdbbf52cb76c7f7771a23ce475f8823f47df 0 7:0 4096
Note the master key which normally shouldn't be leaked, as it breaks whatever brute-force protection LUKS offers. Unfortunately I haven't found a way without using it, as dmsetup
also lacks a direct --make-this-read-write
option. However dmsetup reload
allows replacing a mapping entirely, so we'll replace it with itself in read-write mode.
# dmsetup table --showkeys foobar | dmsetup reload foobar
# dmsetup info foobar
Name: foobar
State: ACTIVE (READ-ONLY)
Read Ahead: 256
Tables present: LIVE & INACTIVE
It's still read-only after the reload, because reload goes into the inactive table.
To make the inactive table active, use dmsetup resume
:
# dmsetup resume foobar
# dmsetup info foobar
Name: foobar
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
And thus we have a read-write LUKS device.
Does it work with a live filesystem?
# cryptsetup luksOpen --readonly foobar.img foobar
# mount /dev/mapper/foobar /mnt/foobar
mount: /mnt/foobar: WARNING: device write-protected, mounted read-only.
# mount -o remount,rw /mnt/foobar
mount: /mnt/foobar: cannot remount /dev/mapper/foobar read-write, is write-protected.
So it's read-only. Make it read-write and remount:
# dmsetup table --showkeys foobar | dmsetup reload foobar
# dmsetup resume foobar
# mount -o remount,rw /mnt/foobar
# echo hey it works > /mnt/foobar/amazing.txt
Can we go back to read-only?
# mount -o remount,ro /mnt/foobar
# dmsetup table --showkeys foobar | dmsetup reload foobar --readonly
# dmsetup resume foobar
# mount -o remount,rw /mnt/foobar
mount: /mnt/foobar: cannot remount /dev/mapper/foobar read-write, is write-protected.
So it probably works. The process to add allow_discards
flag to an existing crypt mapping is similar - you have to reload with a table that contains this flag. However a filesystem that already detected the absence of discard support, might not be convinced to re-detect this on the fly. So it's unclear how practical it is.
Still, unless you have very good reason not to, you should stick to re-opening using regular cryptsetup
commands, even if it means umounting and re-supplying the passphrase. It's safer all around and more importantly, doesn't circumvent the LUKS security concept.
The answer (as I now know): concurrency.
In short: My sequential write, either using dd
or when copying a file (like... in daily use), becomes a pseudo-random write (bad) because four threads are working concurrently on writing the encrypted data to the block device after concurrent encryption (good).
The negative effect can be mitigated by increasing the amount of queued requests in the IO scheduler queue like this:
echo 4096 | sudo tee /sys/block/sdc/queue/nr_requests
In my case this nearly triples (~56MB/s) the throughput for the 4GB random data test explained in my question. Of course, the performance still falls short 100MB/s compared to unencrypted IO.
blktrace
I further investigated the problematic scenario in which a btrfs is placed on a top of a LUKS encrypted block device. To show me what write instructions are issued to the actual block device, I used blktrace
like this:
sudo blktrace -a write -d /dev/sdc -o - | blkparse -b 1 -i - | grep -w D
What this does is (as far as I was able to comprehend) trace IO request to /dev/sdc
which are of type "write", then parse this to human readable output but further restrict the output to action "D", which is (according to man blkparse
) "IO issued to driver".
The result was something like this (see about 5000 lines of output of the multicore log):
8,32 0 32732 127.148240056 3 D W 38036976 + 240 [ksoftirqd/0]
8,32 0 32734 127.149958221 3 D W 38038176 + 240 [ksoftirqd/0]
8,32 0 32736 127.160257521 3 D W 38038416 + 240 [ksoftirqd/0]
8,32 1 30264 127.186905632 13 D W 35712032 + 240 [ksoftirqd/1]
8,32 1 30266 127.196561599 13 D W 35712272 + 240 [ksoftirqd/1]
8,32 1 30268 127.209431760 13 D W 35713872 + 240 [ksoftirqd/1]
This is a snipped of the output produced while dd
'ing the 4GB random data onto the mounted filesystem. It is clear that at least two processes are involved. The remaining log shows that all four processors are actually working on it. Sadly, the write requests are not ordered anymore. While CPU0 is writing somewhere around the 38038416th sector, CPU1, which is scheduled afterwards, is writing somewhere around the 35713872nd sector. That's bad.
blktrace
I did the same experiment after disabling multi-threading and disabling the second core of my CPU. Of course, only one processor is involved in writing to the stick. But more importantly, the write request are properly sequential, which is why the full write performance of ~170MB/s is achieved in the otherwise same setup.
Have a look at about 5000 lines of output in the singlecore log.
Now that I know the cause and the proper google search terms, the information about this problem is bubbling up to the surface. As it turns out, I am not the first one to notice.
Because I (later) found the kernel commit obviously targeted at this exact problem, I wanted to try an updated kernel. [After compiling it myself and then finding out it's already in debian/sid
] It turns out that the problem is indeed fixed. I don't know the exact kernel release in which the fix appeared, but the original commit will give clues to anyone interested.
For the record:
$ uname -a
Linux t440p 4.0.0-1-amd64 #1 SMP Debian 4.0.2-1 (2015-05-11) x86_64 GNU/Linux
$ dd if=/home/schlimmchen/Documents/random of=/mnt/dd-test bs=1M conv=fsync
4294967296 bytes (4.3 GB) copied, 29.7559 s, 144 MB/s
A hat tip to Mikulas Patocka, who authored the commit.
Best Answer
dmsetup
is useful for anything device mapper related. For Example: