dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Encrypted swap crashes all system

Sun Oct 13, 2019 10:18 am

My requirement is to create encrypted swap I need for my project. They project deals with already encrypted data so I don't need full disk encryption just swap. The reason it to protect potential leakage of sensitive information living normally only in memory (like few keys and passwords entered remotely to unlock e.g. rclone config file I am using.) in case of device being stolen.

After a bit of research I thought it should be straightforward but after few days of trying I can't make it working.

My Pi is 3B+ and I run the latest raspbian

Code: Select all

[email protected]:~$ uname -a
Linux dbrpi01 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux
[email protected]:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

I have external disk attached to my Pi with dedicated swap partition (/dev/sda2, 2GB). When I use it this partition as not encrypted swap everything works perfectly. For testing purposes I run stress command (stress -m 1 --vm-bytes 1000M) which immediately forces swap to be used.

Encrypted swap setup uses cryptosetup.

/etc/crypttab file contains following entry

Code: Select all

cryptswap /dev/sda2 /dev/urandom swap,cipher=aes-xts-plain64,size=256

now I can start crypt by running

Code: Select all

sudo cryptdisks_start cryptswap

I can see then that device has been created:

Code: Select all

[email protected]:~$ ll /dev/mapper/
total 0
crw------- 1 root root 10, 236 Oct 13 11:08 control
lrwxrwxrwx 1 root root       7 Oct 13 11:08 cryptswap -> ../dm-0

and I can successfully enable it for swap

Code: Select all

[email protected]:~$ sudo swapon /dev/mapper/cryptswap
[email protected]:~$ sudo swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-0                               partition       2097216 0       -2

So far so good but as soon as swap is being used my Pi simply reboots. I have tried it with two other Pis (3 and 3B+). I have tried to use partition on SD card. To use swap file and loop device. Results are always the same - when encrypted swap is being used device reboots. When swap is not encrypted all works fine.

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 3:45 pm

I tried it on RPi4 (64-bit Ubuntu with kernel 5.3.6), and the encrypted SWAP works perfectly. So there is no reboots when the SWAP is being used, and it is very stable.

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 6:10 pm

Thanks. Have you used partition? Or swap file?

Myself I tried this with latest Debian using VMware and it worked too. Very puzzled.

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 6:14 pm

dariuszb wrote:
Sun Oct 13, 2019 6:10 pm
Thanks. Have you used partition? Or swap file?
Partition on an external HDD, connected through USB 3.0.

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 6:18 pm

Problem is your test is using different OS. Still good to know it is not totally broken. Either my configuration, raspbiab or hardware. The latest least likely as I tried two different systems whit the same results.

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 6:20 pm

Would you mind to try swap file?

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 7:28 pm

dariuszb wrote:
Sun Oct 13, 2019 6:20 pm
Would you mind to try swap file?
I've tried with the following commands:

Code: Select all

mount /dev/sdb1 /mnt
fallocate -l $((4*1024*1024*1024)) /mnt/swapfile
echo "cryptswap /mnt/swapfile /dev/urandom swap,cipher=aes-xts-plain64,size=256" >/etc/crypttab
cryptdisks_start cryptswap
swapon /dev/mapper/cryptswap
stress -m 1 --vm-bytes 5000M
But this time, the system crashed immediately after tried to use the SWAP. And I tried again, but I've got the same result.

However, still good with encrypted partition:

Code: Select all

echo "cryptswap /dev/sdb1 /dev/urandom swap,cipher=aes-xts-plain64,size=256" >/etc/crypttab
cryptdisks_start cryptswap
swapon /dev/mapper/cryptswap
stress -m 1 --vm-bytes 5000M

Code: Select all

> cat /proc/swaps
Filename				Type		Size	  Used	  Priority
/dev/dm-1                               partition	10485756  1871452	-2

dickon
Posts: 529
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 7:42 pm

Do you not need to mkswap any more?

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 7:45 pm

dickon wrote:
Sun Oct 13, 2019 7:42 pm
Do you not need to mkswap any more?
Apparently not.

dickon
Posts: 529
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 7:55 pm

Hmm. News to me. What *isn't* is that fallocate isn't a valid way of doing it: see the mkswap manpage. dd instead.

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 8:08 pm

dickon wrote:
Sun Oct 13, 2019 7:55 pm
Hmm. News to me. What *isn't* is that fallocate isn't a valid way of doing it: see the mkswap manpage. dd instead.
I think the fallocate is good enough on an empty filesystem. But yeah, maybe I'll try with dd too.

dickon
Posts: 529
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 8:12 pm

Not on ext4 (or xfs) it isn't. You're on Raspbian; you're probably hobbled by ext4.

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 8:54 pm

Yes on rasbian. Atm I am not looking for hard core edge. I just want to make it work on general release.

dickon
Posts: 529
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 9:01 pm

Try dd, and I'd recommend mkswap too, because I've yet to see anything that suggests it isn't required, other than assertion above. Perfectly happy to be proven wrong.

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 9:05 pm

dickon wrote:
Sun Oct 13, 2019 8:12 pm
Not on ext4 (or xfs) it isn't. You're on Raspbian; you're probably hobbled by ext4.
It still doesn't work and crashes immediately when the system tries to use SWAP.

Code: Select all

> dd if=/dev/zero of=/mnt/swapfile bs=1M count=4096
> sync
> echo "cryptswap /mnt/swapfile /dev/urandom swap,cipher=aes-xts-plain64,size=256" >/etc/crypttab
> cryptdisks_start cryptswap
 * Starting crypto disk...
 * cryptswap (starting)...
 * cryptswap (started)...                                                [ OK ] 
> mkswap /dev/mapper/cryptswap 
mkswap: /dev/mapper/cryptswap: warning: wiping old swap signature.
Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
no label, UUID=7de51ca9-4e01-466f-b845-1db58e7debfb
> swapon /dev/mapper/cryptswap
> cat /proc/swaps
Filename				Type		Size	Used	Priority
/dev/dm-1                               partition	4194300	0	      -2
> stress -m 1 --vm-bytes 5000M

dickon
Posts: 529
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Encrypted swap crashes all system

Sun Oct 13, 2019 9:09 pm

Fair enough. I'm surprised, but there we go. Always worth ruling out the obvious.

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Tue Oct 15, 2019 10:03 am

I have tried with the latest cryptsetup (version 2.2.1). Results are the same. As soon as system starts swapping it restarts. There is no entry in any log file indicating what happened.

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Tue Oct 15, 2019 10:42 am

Maybe just try with unencrypted SWAP file on an encrypted filesystem. This is one way to encrypts the SWAP, because the entire underlying device/partition is encrypted.

It works for me:

Code: Select all

> dmsetup table --showkeys
data: 0 3906996400 crypt xchacha12,aes-adiantum-plain64 :32:logon:cryptsetup:c4368a29-28ae-4c6b-8746-1045619b272d-d0 0 8:0 32768 2 allow_discards sector_size:4096
> mount | grep /data
/dev/mapper/data on /data type ext4 (rw,noatime)
> fallocate -l $((4*1024*1024*1024)) /data/swapfile
> chmod 0600 /data/swapfile
> mkswap /data/swapfile
Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
no label, UUID=171ea049-c157-41f8-81db-92ea0022d244
> swapon /data/swapfile
> cat /proc/swaps
Filename				Type		Size	Used	Priority
/data/swapfile                          file		4194300	0	      -2
> stress -m 1 --vm-bytes 5000M

Code: Select all

> cat /proc/swaps
Filename				Type		Size	Used	Priority
/data/swapfile                          file		4194300	2123776	      -2

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Tue Oct 15, 2019 11:49 am

Kendek wrote:
Tue Oct 15, 2019 10:42 am
Maybe just try with unencrypted SWAP file on an encrypted filesystem. This is one way to encrypts the SWAP, because the entire underlying device/partition is encrypted.

Have tried - the same reboot. I suspect it must be something to do with 32 bits rasbian and/or limited memory on 3/3B+ raspberries.

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Tue Oct 15, 2019 11:52 am

Also tried with Veracrypt encrypted partition. The same issue.

It would mean that problem is not with cryptsetup but with device mapper (both use it).

sparkyhall
Posts: 145
Joined: Mon Aug 27, 2012 9:14 am

Re: Encrypted swap crashes all system

Tue Oct 15, 2019 6:34 pm

I wouldn't expect a full reboot, feels more like a hardware/power issue to me.

Or are you saying it crashes or locks up?

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Tue Oct 15, 2019 6:40 pm

sparkyhall wrote:
Tue Oct 15, 2019 6:34 pm
I wouldn't expect a full reboot, feels more like a hardware/power issue to me.

Or are you saying it crashes or locks up?
Reboot. Without swap encryption everything works perfectly. I use it hard and never any issues. And it happens with other hardware too.. I tried before even posting here. Only what is in common is encrypted swap on generic raspbian.

dariuszb
Posts: 33
Joined: Sun Feb 21, 2016 3:55 pm

Re: Encrypted swap crashes all system

Tue Oct 15, 2019 6:51 pm

There is nothing in logs. no OOM or anything. Only entries that encryption in initialized, swap made and mounted. then next entry is beginning of next startup.

I use encrypted disks for other data purposes without any problems. It is only when it is swap. I did tones of google but without any success to find some clues. I saw few people acknowledging that encrypted swap files don't work (apparently they did with ancient kernels) so I try to make encrypted partition. However unless somebody sheds some light on this I think I hit the wall.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23901
Joined: Sat Jul 30, 2011 7:41 pm

Re: Encrypted swap crashes all system

Thu Oct 17, 2019 11:21 am

Thought I would give this a try, but fails at the

sudo cryptdisks_start cryptswap

Is there a problem with using ext4 (or vfat)?

[warn] Starting crypto disk...cryptswap (starting)...[....] cryptswap: the precheck for '/dev/mmcblk0p2' failed: - The device /dev/mmcblk0p2 contains a filesystem type ext4. ... (warning).
failed.

If it takes much more effort than this to test it I probably won't bother - it's such a small use case I don't want to spend much time on it.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Kendek
Posts: 127
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Encrypted swap crashes all system

Thu Oct 17, 2019 11:30 am

jamesh wrote:
Thu Oct 17, 2019 11:21 am
Thought I would give this a try, but fails at the
The simple steps:

Code: Select all

> fallocate -l $((4*1024*1024*1024)) /swapfile
> echo "cryptswap /swapfile /dev/urandom swap,cipher=aes-xts-plain64,size=256" >>/etc/crypttab
> cryptdisks_start cryptswap
 * Starting crypto disk...                                                      
 * cryptswap (starting)...                                                      
 * cryptswap (started)...                                                [ OK ] 
> swapon /dev/mapper/cryptswap
> cat /proc/swaps
Filename				Type		Size	Used	Priority
/dev/dm-1                               partition	4194300	0	-2

Return to “Advanced users”