Adding extra RAM

[ech0]

What about turning an USB flash drive into some extra virtual memory?

Would that work with the RPi, what do you guys say?

ech0

[r3d4]

not realy ram tho

[Ravenous]

The question's come up before, and I gather it would be limited in speed by the USB communication so not really worthwhile.  It would be more like having extra swap space available.

[jamesh]

Just use the SD card as swap space.

[Jim Manley]

JamesH said:

Just use the SD card as swap space.



USB 2.0 hard and solid-state drives will be up to an order of magnitude faster (40 ~ 100 MB/s) than typical SD cards (4 ~ 6 MB/s for Class 6), and won"t suffer the longevity limitations inherent in SD cards due to the engineering assumption the latter will be written to infrequently. Even some USB flash thumb drives can outperform Class 6 SD cards by at least a factor of two (although some will suffer capacity degradation). Sure, the SD card will last a few months, or so, before starting to degrade in terms of capacity as the most frequently-written address locations fail, but, who needs the hassle of recopying a card every so often?

[Justanick]

Also a SD card has an algorithm for LBA and flash address mapping.

If you write every time to sector 1 on the flash device on the Pi you will *not* always hit the same flash address.

The same is valid for USB stick.

[toxibunny]

A USB HD is definitely the way to go for swap. It"d be nice if USB RAM (not flash) sticks were available, but it seems they don"t exist...

[Justanick]

Get a cheap stick and burn it down, who cares …

The access time may be get important to reload the swap back to ram under heavy load.

[Jim Manley]

Justanick said:

Also a SD card has an algorithm for LBA and flash address mapping.

If you write every time to sector 1 on the flash device on the Pi you will *not* always hit the same flash address.

The same is valid for USB stick.


Even with LBA, swap can get written to a _lot_, especially with limited RAM systems like the R-Pi, and is completely rewritten at each boot. Cheaper/older/smaller flash memory only has an average life in the hundreds of thousands of writes, and some addresses will start failing with as few as a few thousand writes, statistically (although some will last for many millions of writes). LBA takes time, that"s why flash devices are so slow relative to other technologies. Flash devices generally don"t have significant RAM buffers like hard and solid-state drives, and that makes a huge difference - some buffers are as large as the entire R-Pi RAM space.

[jojopi]

Jim Manley said:

USB 2.0 hard and solid-state drives will be up to an order of magnitude faster (40 ~ 100 MB/s) than typical SD cards (4 ~ 6 MB/s for Class 6)
What matters for swap is the access latency or I/O operations per second, especially for small (4K) random writes, rather than the sustained MB/s.  But this may still favour swapping to USB, especially if you compare the best UFDs or SSDs to the best SDs.

For completeness, swapping to network can also be made to work.  But all swap targets on a Pi will be relatively high latency, so conserving memory will be more of a magic bullet.

[Justanick]

Jim Manley said:


Justanick said:


Also a SD card has an algorithm for LBA and flash address mapping.

If you write every time to sector 1 on the flash device on the Pi you will *not* always hit the same flash address.

The same is valid for USB stick.


Even with LBA, swap can get written to a _lot_, especially with limited RAM systems like the R-Pi, and is completely rewritten at each boot. Cheaper/older/smaller flash memory only has an average life in the hundreds of thousands of writes, and some addresses will start failing with as few as a few thousand writes, statistically (although some will last for many millions of writes). LBA takes time, that"s why flash devices are so slow relative to other technologies. Flash devices generally don"t have significant RAM buffers like hard and solid-state drives, and that makes a huge difference - some buffers are as large as the entire R-Pi RAM space.


The most of the Ram on the SSD are using for the mapping tables and not for cache for write data.

Try it with a stick. A cheap 2GB free gift is doing the swap job and as SVN cache for more than 2 years on the router with dd-wrt.

[Jim Manley]

Justanick said:


The most of the Ram on the SSD are using for the mapping tables and not for cache for write data.

Try it with a stick. A cheap 2GB free gift is doing the swap job and as SVN cache for more than 2 years on the router with dd-wrt.


Just out of curiosity, did you happen to record the capacity of the cheap 2GB stick down to the byte when you started using it, and what it is now? It will be different by MBs. A router application has very different behavior from a maxed-out-RAM, multi-tasking Linux system with at least one user who"s scrolling through documents, caching significant-sized web pages, watching videos, listening to audio files/streams, and literally hundreds of subprocesses supporting all of the above, including anti-virus scanning, memory management, network management, process management. Start running enough apps and services to max out your memory to force swapping, set up "top" running in a window and watch things bouncing around as you go about your daily routine and you"ll get an idea about how how hard life is for your swap partition.

How often is a router getting page misses? Not many if you have the right amount of RAM – it depends on what the traffic load is. It will be a lot different for a small office group LAN router moving documents, e-mail, and a few streams vs. a large site WAN interface carrying tons of video streams, VOIP sessions, e-mail (mostly spam not seen on LANs), lots of file transfers, and who-knows-what-else.

For the SVN cache, how many users is it supporting, how many files are changed on each update, how often are updates done, how big are the changes that are made, and how big are the files? Other than establishing a new local copy, the changes are very small and relatively few in number compared to network services like e-mail, web pages, file transfers, video and audio streams (always new data), VOIP, etc.

SSDs only use the latest, largest flash devices which have the best longevity (which has been improving with each generation) and made to the highest quality standards. They do have RAM data buffers because of LBA – even with Class 10 devices they would be way too slow to replace hard drives without buffers, as even the slowest 5,400 RPM drives have better transfer rates than the most expensive, non-custom SSDs (and guess what the most expensive SSDs have – lots of the fastest RAM data buffers). Flash devices also have widely-differing performance with different block and file sizes. Reconfigure a flash drive with a variety of block sizes and then transfer various sizes of files, particularly over time so that things get nice and fragmented into chunks bigger than the block size and data buffer capacity. They can get positively molasses-in-January slow in the worst-case mixes.

As they say in the commercials, YMMV … widely.

[nick.mccloud]

Perhaps to put this in context, particularly the desire to do some serious multi-tasking - the actual hands on feedback is that the Pi is well up to the job it's designed for (kids programming workstation or light general single-tasking usage or video playback or Quake) but it is not up to replacing a busy working GUI environment with word processing, web browsing, email, chat client, etc etc all going on at once (see Chris Tyler's comments on running Fedora GUI).

Once we have some to play with all sorts of combo's can be tried, including putting the majority of the file system on a small HD with as much swap as you want but it's still going to be constrained with the USB interface - particularly if you are doing some serious download as the ethernet is routed via USB as well. Until that time we are sort of guessing.

As I can grind my MacBook i5 with 8Gb to a crawl by lunchtime by littering it with open apps, some of them background processing, I shudder to think what I can do to a Pi.

But as ever, YMMV

[rew]

Jim,

Your idea about how USB sticks work is wrong.

There is an interface specification. This explains what a computer can expect from a "harddisk" like device (like an USB stick) and what the USB stick can expect from the computer.

In the old days, the interface specification simply included that "some blocks are bad: they don't work. Deal with it!". So operating systems are built to handle that case. However nowadays the interface specification has been changed to say that the harddisk(-like-object) presents an array of working blocks and handles any non-working blocks itself. The operating systems haven't been modified, and are still able to work with imperfect harddisks.

Anyway… Harddisks and Flash drives therefore have to PRETEND that everything is perfect, while in fact it isn't. So harddisks and flash drives remap bad sectors. They start out with some "spare space", and use that if they determine that a block is going bad.

So…. most flash chips have 520 bytes of storage for every 512 bytes they say they have. These 8 bytes are enough to enable the controller of the flash chip to reconstruct a flipped bit. So when a block is read and the controller needs to correct a bit error it could mark the block as "bad" and use one of the spare ones instead.All this happens outside of the view of the operating system. This is done INSIDE the USB flash drive (or harddisk).

Have you ever checked the size in bytes of your 32Gb flash drive? It is about 32 billion bytes: 32000000000. Have you ever asked a computer scientist how many bytes 32Gb is? He'll tell you 2^35 bytes: 34359738368.

They don"t make 32000000000 byte flash chips. In fact they are 520/512*2^35 bytes. So there is over 2Gb of flash bytes available for remapping and wear levelling. Those bytes are kept hidden from the computer right from the start.

Now, the certainty with which you're stating these non-facts, lead me to believe that you've actually seen this happen with your own eyes. I'm pretty sure there is an explanation for what you've seen, but it's not that the bad blocks get subtracted from the "available space" on the drive.

For example, I've had an USB-micro-SD cardreader that had a bug. It would report the size of the PARTITION as the size of the device. So I'd partition and  format it and all would be fine. Remove and reinsert and an all of a sudden: "device is too small to hold partition". Partition and reformat. Ten minutes later the device became smaller again! Took me a while to figure out.

Another thing you might be seeing is that if you format a flash drive with the NTFS filesystem, the so called MFT (master file table) can only grow. So if you add a lot of files and then remove them the MFT will have taken more space to hold all the info about that "lot of files", but that space cannot be reclaimed. So in the end you'll see less available space, even though you've removed all those files.

[lesliew]

rew said:


In the old days, the interface specification simply included that "some blocks are bad: they don't work. Deal with it!". So operating systems are built to handle that case. However nowadays the interface specification has been changed to say that the harddisk(-like-object) presents an array of working blocks and handles any non-working blocks itself. The operating systems haven't been modified, and are still able to work with imperfect harddisks.

Anyway… Harddisks and Flash drives therefore have to PRETEND that everything is perfect, while in fact it isn't. So harddisks and flash drives remap bad sectors. They start out with some "spare space", and use that if they determine that a block is going bad.

So…. most flash chips have 520 bytes of storage for every 512 bytes they say they have. These 8 bytes are enough to enable the controller of the flash chip to reconstruct a flipped bit. So when a block is read and the controller needs to correct a bit error it could mark the block as "bad" and use one of the spare ones instead.All this happens outside of the view of the operating system. This is done INSIDE the USB flash drive (or harddisk).

Have you ever checked the size in bytes of your 32Gb flash drive? It is about 32 billion bytes: 32000000000. Have you ever asked a computer scientist how many bytes 32Gb is? He'll tell you 2^35 bytes: 34359738368.

They don"t make 32000000000 byte flash chips. In fact they are 520/512*2^35 bytes. So there is over 2Gb of flash bytes available for remapping and wear levelling. Those bytes are kept hidden from the computer right from the start.



Very informative, Thanks

[daveslee]

ech0 said:


What about turning an USB flash drive into some extra virtual memory?

Would that work with the RPi, what do you guys say?


Perhaps putting a solid state drive on the end of USB?

But first, can you tell us about your potential application for the Raspberry-Pi and how adding a whole pile of extra processor main memory mapped RAM helps the application to be possible instead of impossible? I've de-lurked because I see so many people saying can I have more of this resource or that resource; the answer is of course yes, but it has a cost. If all you are looking for is a cheap general purpose computer then perhaps the Rasp-Pi is the wrong place to look - go buy a laptop PC and throw away the case. Or sell a software product on a general purpose platform.

I'm deliberatley being a bit provocative. My undertanding of the charitable aims of the Foundation are to promote good solid coding on resource constrained platforms and a return to the renaissance years of computing where elegance and efficiency were prized.

So tell us more about your application(s) and perhaps we can talk about how to realise your application in an efficient and elegant way rather than throwing RAM and processor cycles at it. Don't get me wrong - for some applications pure brute force is the only way, but can I suggest for the majority od need that there is a smart and clever way based on real computer science - an algorithmic understanding of your problem/application.

That's what I want the Rasp-Pi to bring back to the community. Not Oliver Twist: "can I have some more please?", but instead the entreprenurial thinking "What can I squeeze out of how little I have got?"