tawalker
Posts: 183
Joined: Tue Jan 17, 2012 9:02 am
Contact: Website

Re: Swap space on an SD card

Wed Feb 01, 2012 9:54 am

Apologies if this has been posted about before (I did carry out a small search, but didn't find much), but I was wondering about the issue of having a swap partition on an SD card?

Basically, the received wisdom as I understand it, is that a swap partition on flash memory is not generally advisable, as it involves lots of write operations which can wear out flash cells more quickly. (I also understand that apparently, modern flash memory and/or controllers have features which can mitigate this so it's less of a problem ("wear-levelling"?).)

On my Eee 701SD netbook (Arch Linux), to minimise wear on the SSD, I don't have a swap partition on the drive, and have moved the temp files and logging to a ramdisk (I can get away with this, having upgraded the machine's RAM to 2GB).

However, I don't think this is an option on the RasPi, owing to its lower RAM (256MB on the Model B). Presumably it will need a swap partition, and this would need to be on the SD card. Does this mean I'll just have to assume that the card will wear out more quickly, and therefore be careful to back it up and reflash the contents onto a new card, when the old one "dies"?

Many thanks
---
Raspberry Pi Model 3B+ (2019) ("ayeka") - CentOS
Raspberry Pi Zero W (2018) ("mass") - Raspbian
Raspberry Pi Model B (1st-gen - 2012) ("ryo-ohki") - Arch Linux ARM
---

User avatar
RaTTuS
Posts: 10539
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Swap space on an SD card

Wed Feb 01, 2012 10:02 am

you dont have to have a swap , and if your doing heavy stuff which needs swap you can always add a HD via the USB -
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

mank0r
Posts: 6
Joined: Tue Jan 24, 2012 12:35 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 10:04 am

You're on the right track. If you prefer, there should be no problem connecting a USB memory stick to the pi, and using that for swap, temp, and log files. You should be fine using a swap file rather than a swap partition as well, meaning everything can live on the same partition. You'll still face the issue if flash wear unless you move to an HDD or a more expensive SSD (the speed benefits are wasted) or USB stick.

plugwash
Forum Moderator
Forum Moderator
Posts: 3546
Joined: Wed Dec 28, 2011 11:45 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 10:11 am

As well as the wear issues swap on an SD card is going to be EXCRUCIATINGLY slow (SD cards SUCK at random write access). If your application can't live within the available memory I would strongly suggest either making external arrangements for swap or using something other than a Pi.

kme
Posts: 448
Joined: Sun Sep 04, 2011 9:37 am

Re: Swap space on an SD card

Wed Feb 01, 2012 10:16 am

plugwash said:


As well as the wear issues swap on an SD card is going to be EXCRUCIATINGLY slow (SD cards SUCK at random write access). If your application can't live within the available memory I would strongly suggest either making external arrangements for swap or using something other than a Pi.


+1

bredman
Posts: 1415
Joined: Tue Jan 17, 2012 2:38 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 10:45 am

I agree with mank0r to put swap on a cheap USB flash drive.

Form personal experience, I use an old 256MB USB flash drive as a swap partition on my home-built firewall which runs 24/7.

The activity light is regularly flashing to show that the swap is in use, and it hasn't failed yet after 2 years of use.

hyena
Posts: 44
Joined: Mon Nov 14, 2011 7:55 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 10:51 am

dont all these sd's/usb sticks have wear levelling built in now ?  ..

redman684
Posts: 48
Joined: Fri Dec 09, 2011 5:25 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 11:57 am

Wear leveling (which means the hardware will relocate blocks of data so every part of the memory is written equally often) seems indeed common place, although the systems vary and you never know with shady hardware whether there is any wear leveling at all.

I often see memory that can be written 100,000 times and even claims of over 1,000,000 are made.

Writing an 8 GB card (with perfect wear leveling) 100,000 times with 10 MB/s will take 2.6 years of continuous writing. So it's pretty safe to say the card won't wear out anytime soon, even when used as swap space.

User avatar
Burngate
Posts: 6224
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: Swap space on an SD card

Wed Feb 01, 2012 12:49 pm

Yes, but fill that 8 GB card with 8191 MB of unchanging data, then write the last 1 MB 100,000 times! At 10 MB/s it'll take only 10,000 secs = < 3 hours. And wear-leveling couldn't work with no free space, so it could die in even less time!

rpt
Posts: 51
Joined: Tue Jan 31, 2012 3:09 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 12:55 pm

Burngate said:


Yes, but fill that 8 GB card with 8191 MB of unchanging data, then write the last 1 MB 100,000 times! At 10 MB/s it'll take only 10,000 secs = < 3 hours. And wear-leveling couldn't work with no free space, so it could die in even less time!


But with wear-levelling it won't keep writing to the last 1MB of physical flash but will spread that over the whole device. So 2.6 years sounds right. Is that the expected life expectancy of high tech devices these days?

WereCatf
Posts: 89
Joined: Thu Jan 26, 2012 7:16 am
Contact: Website

Re: Swap space on an SD card

Wed Feb 01, 2012 12:59 pm

tawalker said:

Basically, the received wisdom as I understand it, is that a swap partition on flash memory is not generally advisable, as it involves lots of write operations which can wear out flash cells more quickly. (I also understand that apparently, modern flash memory and/or controllers have features which can mitigate this so it's less of a problem ("wear-levelling"?).)
Wear-levelling is a process where the controller keeps a constant count on how many times every cell has been written to, and tries to keep things so that all cells have equal writes to them. Commonly Flash-media like SD-cards and USB-sticks do not have wear-levelling and on such devices swap may wear out the device. On SSDs though you will always find wear-levelling, and on many kinds of internal Flash-technologies that are usually not available to the wider, general customer base.


On my Eee 701SD netbook (Arch Linux), to minimise wear on the SSD, I don't have a swap partition on the drive, and have moved the temp files and logging to a ramdisk (I can get away with this, having upgraded the machine's RAM to 2GB).


As said above, SSDs do have wear-levelling and as such having swap on an SSD isn't really a problem. Also, just the fact that there does exist a swap partition doesn't necessarily mean it's being used; your OS only uses it anyways if it seems to be running out of RAM, and if no such condition exists then there are no writes to the SSD.


However, I don't think this is an option on the RasPi, owing to its lower RAM (256MB on the Model B). Presumably it will need a swap partition, and this would need to be on the SD card. Does this mean I'll just have to assume that the card will wear out more quickly, and therefore be careful to back it up and reflash the contents onto a new card, when the old one "dies"?


The whole drive doesn't suddenly die if some cells die, meaning that you could e.g. buy a 16GB card, make a 14GB root partition for the OS, make a 1G swap partition at the 14GB->15GB mark and use that until it starts showing signs of corruption, then make a second swap partition at the 15GB-16GB mark and use that and so forth.

Also as I have used Flash media for all kinds of purposes for years now I would say that you could safely use it atleast for a year straight before you start getting issues. Atleast I have been able to use Flash media very heavily for a good, long time before it has started to show any kinds of symptoms, and with "very heavily" I do mean much more write-heavy tasks than simple swap.

WereCatf
Posts: 89
Joined: Thu Jan 26, 2012 7:16 am
Contact: Website

Re: Swap space on an SD card

Wed Feb 01, 2012 1:06 pm

rpt said:


Burngate said:


Yes, but fill that 8 GB card with 8191 MB of unchanging data, then write the last 1 MB 100,000 times! At 10 MB/s it'll take only 10,000 secs = < 3 hours. And wear-leveling couldn't work with no free space, so it could die in even less time!


But with wear-levelling it won't keep writing to the last 1MB of physical flash but will spread that over the whole device. So 2.6 years sounds right. Is that the expected life expectancy of high tech devices these days?



It depends on what you mean with "high tech devices." SSDs with MTBF of 1.5M or more should last for about 10 years on regular desktop and gaming uses, though for heavy video-editing tasks one should instead get one of the more expensive SSDs with high-end cells. I do not have any estimates about SD-card lifetimes to offer, though.

WereCatf
Posts: 89
Joined: Thu Jan 26, 2012 7:16 am
Contact: Website

Re: Swap space on an SD card

Wed Feb 01, 2012 1:11 pm

Burngate said:


Yes, but fill that 8 GB card with 8191 MB of unchanging data, then write the last 1 MB 100,000 times! At 10 MB/s it'll take only 10,000 secs = < 3 hours. And wear-leveling couldn't work with no free space, so it could die in even less time!



On SSDs there is usually something around 10% of the space actually reserved for exactly this reason: if a cell dies completely the controller marks it as dead and takes one from the reserve to work on instead. This works even if the drive is full as the reserved area is exactly that: reserved. It cannot be written to, it cannot be read from and it cannot be accessed in any way or form other than indirectly when the controller decides so.

On SD-cards and similar there however is no reserved area and so far I have not seen a single SD-card with wear-levelling either, so on such devices what you're saying could indeed well happen.

redman684
Posts: 48
Joined: Fri Dec 09, 2011 5:25 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 2:05 pm

Burngate said:


Yes, but fill that 8 GB card with 8191 MB of unchanging data, then write the last 1 MB 100,000 times! At 10 MB/s it'll take only 10,000 secs = < 3 hours. And wear-leveling couldn't work with no free space, so it could die in even less time!


That's trying to find a way to break the memory asap. It's not realistic, nor usefull (write but never read?) And a wear leveling can very well relocate an existing piece of memory. It would be silly to just wear level unused space.

The kind of wear leveling applied (if any) will probably differ between brands and even models.

I see conflicting sources. Some say wear leveling is used on SD (and even micro SD), some say it's part of the SD standard, but others say their products wear leveling commands are only supported (but ignored) to provide backwards compatibility. Whether that means wear leveling is not used, or became completely transparent and no longer needs external commands, it not clear to me.

Wear leveling has certainly been available on SD, but I don't know it's current state. It would surprise me if it has been removed.

User avatar
jojopi
Posts: 3142
Joined: Tue Oct 11, 2011 8:38 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 2:07 pm

WereCatf said:

On SD-cards and similar there however is no reserved area and so far I have not seen a single SD-card with wear-levelling either, so on such devices what you're saying could indeed well happen.
SD cards do have reserved space -- around 7% if the ones I have here are typical.  And they do implement hardware dynamic wear levelling, but this is not as good as the static wear levelling normally provided on SSDs.

The only flash formats with no wear levelling at all were the controller-less SmartMedia and xD-Picture Card.  These needed special filesystems to avoid wearing out fixed metadata areas like the FAT.

At ~$10 for 8GB I do not think it is worth worrying about anyway.  Maybe I have been lucky, but I have never worn out any flash medium before it became obsolete due to Moore's law.

monthos
Posts: 6
Joined: Sun Jan 22, 2012 6:42 pm

Re: Swap space on an SD card

Wed Feb 01, 2012 2:14 pm

I killed a couple compact flash cards back in the day working with linux on a Jornada 680. I eventually moved to using a swap file mounted through NFS on another system in my network. Slow, but it stopped eating cards after that.

WereCatf
Posts: 89
Joined: Thu Jan 26, 2012 7:16 am
Contact: Website

Re: Swap space on an SD card

Wed Feb 01, 2012 2:17 pm

jojopi said:


WereCatf said:


On SD-cards and similar there however is no reserved area and so far I have not seen a single SD-card with wear-levelling either, so on such devices what you're saying could indeed well happen.


SD cards do have reserved space -- around 7% if the ones I have here are typical.  And they do implement hardware dynamic wear levelling, but this is not as good as the static wear levelling normally provided on SSDs.



Perhaps I've then just always gotten wrong kinds of cards, but whenever I've been testing and using them they've always worked as if there was no wear-levelling whatsoever. You keep writing to the same spot and it wears out, even if the rest of the card was completely unused. As such I personally wouldn't count on SD-cards having any meaningful wear-levelling. Perhaps it only works on FAT, atleast I saw a mention of such a scheme on some manufacturers' cards, whereas it would be much more useful if it was filesystem-agnostic.

tawalker
Posts: 183
Joined: Tue Jan 17, 2012 9:02 am
Contact: Website

Re: Swap space on an SD card

Fri Feb 03, 2012 10:41 am

Thanks for all the feedback on my original post - much appreciated! I think I may have to take an image "snapshot" of the SD card every now and then, in case I have to replace a worn-out card once in a blue moon...
---
Raspberry Pi Model 3B+ (2019) ("ayeka") - CentOS
Raspberry Pi Zero W (2018) ("mass") - Raspbian
Raspberry Pi Model B (1st-gen - 2012) ("ryo-ohki") - Arch Linux ARM
---

redman684
Posts: 48
Joined: Fri Dec 09, 2011 5:25 pm

Re: Swap space on an SD card

Fri Feb 03, 2012 4:38 pm

A backup is never a bad idea. Although I do not expect an SD-card to wear out anytime soon, it can fail due to many other reasons, varying from a lesser quality card/batch up to a lightning strike. Or you might have perfectly functioning hardware, but accidentally overwrite/corrupt your data, get a virus, etc.

You do not need to be scared the card can break down any minute, but it could fail, just like any other device.

Docteh
Posts: 32
Joined: Tue Jan 31, 2012 6:20 am

Re: Swap space on an SD card

Sun Feb 05, 2012 9:23 am

Heh, just a few months ago i was joking about using an ssd in a usb enclosure. Might have a legit reason!

rrolsbe
Posts: 40
Joined: Fri Aug 12, 2011 4:09 pm

Re: Swap space on an SD card

Wed Feb 08, 2012 7:43 pm

I concur that all modern USB/SDHC flash devices have reserved Nand blocks but AFAIK they only use these reserved blocks to replace failing blocks and NOT to improve write performance as some of their SSD counterparts with large reserved Nand blocks do.  Unless the controllers on these low cost USB/SDHC flash devices supported some form of pre-erase in the reserved area, using reserved blocks would have no performance benefit after the Nand pages in the reserved area were written the first time; however, since static wear leveling is not normally supported using these spare blocks might extend the life of the flash device.  If the file system partition on an SDHC flash device were filled with files with static file content and a Linux SWAP file was created that used most/all of the remaining area, the flash in the SWAP file area would probably wear out fairly quick (ie.. wear leveling would not be performed over the entire flash device usable Nand blocks (Static wear leveling) but only use the remaining Non-Static Nand page area (Dynamic wear leveling...). I use a Swap file on the SDHC card in my OLPC computer only to keep the OS from locking up if I exhaust the 256MB of internal SDRAM and NOT to improve performance because the SDHC writes are VERY slow.  Since both the Raspberry Pi and the OLPC version 1.0 run the OS from SCHC flash devices up to 32G in size (maybe 64GB?), it would be nice if part of that 32G SDHC device could be mapped/addressed to use SDRAM.  If an area of the SDHC device could be mapped to use SDRAM, then read/write performance to that area could approach USB 2.0 data rates approx 50MB/sec.  Of course this would require a redesign of the SDHC interface and the SDHC devices we use today and creation of a swap partition in the SDRAM area each time the computer was booted.  I will probably use a Linux Swap file on an externally attached/powered SSD but the Read/Write performance will be USB 2.0 (which is much lower that my SSD but should be considerably faster than writing to SDHC flash.)

Regards, Ron

boarsblood
Posts: 29
Joined: Mon Jan 30, 2012 8:13 pm

Re: Swap space on an SD card

Wed Feb 08, 2012 9:19 pm

You just need to find an old Elan Mobidaptor then use it to adapt a usb hard drive to sd interface, plug them into the R-Pi's sd slot and you won't have to worry about Flash memory wearing out.

http://www.elandigitalsystems......dapter.php

In the interests of fairness other usb drive to sd card adaptors are available

rrolsbe
Posts: 40
Joined: Fri Aug 12, 2011 4:09 pm

Re: Swap space on an SD card

Wed Feb 08, 2012 10:02 pm

boarsblood said:


You just need to find an old Elan Mobidaptor then use it to adapt a usb hard drive to sd interface, plug them into the R-Pi's sd slot and you won't have to worry about Flash memory wearing out.

http://www.elandigitalsystems......dapter.php

In the interests of fairness other usb drive to sd card adaptors are available


Looks interesting but $59 is pricey for an adapter.  Do you know who else makes such a device?

http://www.teampctechnology.co.....hp?id=603#

boarsblood
Posts: 29
Joined: Mon Jan 30, 2012 8:13 pm

Re: Swap space on an SD card

Thu Feb 09, 2012 1:06 am

rrolsbe said:


boarsblood said:


You just need to find an old Elan Mobidaptor then use it to adapt a usb hard drive to sd interface, plug them into the R-Pi's sd slot and you won't have to worry about Flash memory wearing out.

http://www.elandigitalsystems......dapter.php

In the interests of fairness other usb drive to sd card adaptors are available


Looks interesting but $59 is pricey for an adapter.  Do you know who else makes such a device?

http://www.teampctechnology.co.....hp?id=603#



Not that shows up on a quick FWSE. I dealt with Elan a few years ago and used some of their sd-usb adaptor, great product, they didn't update them to support bigger storage because there was no demand, so it died a death and most people today think an SD input to USB device adaptor is impossible; rather then awkward. Now there's a demand but no product, maybe they'd license their code for a community buy-out as they're not using it.

rrolsbe
Posts: 40
Joined: Fri Aug 12, 2011 4:09 pm

Re: Swap space on an SD card

Fri Feb 10, 2012 9:39 pm




Bus Interface
Card Type
Bus Mark
Bus Speed
Spec Version


Normal Speed
SD, SDHC and SDXC
---
12.5MB/s
1.01


High Speed
SD, SDHC and SDXC
---
25MB/s
2.00


UHS-I
SDHC and SDXC

50MB/s (SDR50, DDR50)
104MB/s (SDR104)
3.01


UHS-II
SDHC and SDXC
TBD
156MB/s
312MB/s
4.00




So what version SDHC reader is the Pi using? I did not realize the SDHC NON UHS-I readers/cards had a bus speed of only 25MB/sec.

rrolsbe said:


I concur that all modern USB/SDHC flash devices have reserved Nand blocks but AFAIK they only use these reserved blocks to replace failing blocks and NOT to improve write performance as some of their SSD counterparts with large reserved Nand blocks do.  Unless the controllers on these low cost USB/SDHC flash devices supported some form of pre-erase in the reserved area, using reserved blocks would have no performance benefit after the Nand pages in the reserved area were written the first time; however, since static wear leveling is not normally supported using these spare blocks might extend the life of the flash device.  If the file system partition on an SDHC flash device were filled with files with static file content and a Linux SWAP file was created that used most/all of the remaining area, the flash in the SWAP file area would probably wear out fairly quick (ie.. wear leveling would not be performed over the entire flash device usable Nand blocks (Static wear leveling) but only use the remaining Non-Static Nand page area (Dynamic wear leveling...). I use a Swap file on the SDHC card in my OLPC computer only to keep the OS from locking up if I exhaust the 256MB of internal SDRAM and NOT to improve performance because the SDHC writes are VERY slow.  Since both the Raspberry Pi and the OLPC version 1.0 run the OS from SCHC flash devices up to 32G in size (maybe 64GB?), it would be nice if part of that 32G SDHC device could be mapped/addressed to use SDRAM.  If an area of the SDHC device could be mapped to use SDRAM, then read/write performance to that area could approach USB 2.0 data rates approx 50MB/sec.  Of course this would require a redesign of the SDHC interface and the SDHC devices we use today and creation of a swap partition in the SDRAM area each time the computer was booted.  I will probably use a Linux Swap file on an externally attached/powered SSD but the Read/Write performance will be USB 2.0 (which is much lower that my SSD but should be considerably faster than writing to SDHC flash.)

Regards, Ron


Return to “General discussion”