RootFS on USB Flash Drive


38 posts   Page 1 of 2   1, 2
by rmm200 » Fri May 25, 2012 7:46 pm
I really hate starting a topic on something I am sure has been covered, but I have searched the Forum and the WIKI.
Can someone post a link to a walkthrough on how to set up a boot sd card, and a rootfs on a USB Flash Drive.
Newark is finally about to ship my unit and I need to get ready...
Thanks!
Posts: 259
Joined: Sat Mar 03, 2012 10:25 pm
by jbeale » Fri May 25, 2012 8:08 pm
Here are two threads on the subject:
viewtopic.php?f=7&t=5695
viewtopic.php?t=5574&p=76496

I don't have a Pi yet myself but 'rogerdean' reports success in this post (below shown edited for clarity)
viewtopic.php?f=24&t=5695#p76810

Re: Booting from the SD card, but using a USB stick to run the OS & store the file system
Post by rogerdean » 12 May 2012 13:49

Ok, I've done it. The secret was, as in so many places on these forums, that my power supplies (cellphone chargers) just weren't giving enough power for the portable hard drive. I had 3, one powering the Pi, one powering the USB hub, and one powering the hard drive through a split cable. Not good enough.

Then I tried using my laptop USB port as an additional power supply for the hard disk, and all was well. So, the steps are -

1) find a way to get enough power to the USB hard drive
    many cell phone chargers don't provide enough current
2) write the Debian squeeze image to both hard drive and to your SD card
3) on the root partition of the SD card, modify cmdline.txt to point to the root partition on the USB device
    if your root partition is sda2, then edit the cmdline.txt to show root =dev/sda2 instead of root=mmcblk0p2
    the root partition is usually the smallest one, about 78 MB in size
4) on the root partition of the SD card, replace kernel.img with the new one from github
5) plug it all in and off you go!

It's hugely faster, feels more like a real (very old) computer. Good luck
Last edited by jbeale on Fri May 25, 2012 8:34 pm, edited 2 times in total.
User avatar
Posts: 2084
Joined: Tue Nov 22, 2011 11:51 pm
by AndrewS » Fri May 25, 2012 8:27 pm
Don't have a Raspi myself (yet), so I can only provide high-level instructions, rather than a fully detailed walkthrough, but AFAIK the rough steps involved are something like:
    extract your preferred distro image to an SD card - plenty of instructions for this elsewhere
    mount the rootFS partition from the SD card (usually partition 2) - on most linux desktops easiest way is just to (safely) remove and reinsert the card
    format your USB drive to ext3 or ext4 as appropriate/preferred
    copy the entire filesystem from the SD card partition2 (I'll assume /media/rootfs here) to your USB drive partition1 (I'll assume /media/volume1 here). Double-check that you get the directory names the right way round!
    Code: Select all
    sudo rsync -avxS /media/rootfs /media/volume1
    Edit the cmdline.txt in the SD card's boot partition (partition 1) and change
    Code: Select all
    root=/dev/mmcblk0p2
    (the second partition on the SD card) to
    Code: Select all
    root=/dev/sda1
    (the first partition on the USB drive)
    plug the SD card and USB drive into the Raspberry Pi and power up :)

EDIT: Doh, beaten to it! But my "instructions" don't create a redundant partition on the USB drive, and don't require the rootfs partition to be resized afterwards ;)
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
by Joe Schmoe » Fri May 25, 2012 8:36 pm
You don't actually have to write it out to the SD card; you can just mount the image using the loop device and work off of the image. There are a couple of advantages to this:

1) (Obvious) Faster/fewer steps/better performance reading from a disk file than from a slow device.

2) Don't have to "crud up" the SD card with partitions and stuff. I.e., all you have to do is copy the files from the FAT partition in the image to the SD card - no reformatting or stuff like that.

2a) The SD card can be any size you like - i.e., small. Doesn't have to be at least 2G or so.

Note: Even though this post is written in "hillbilly tech speak", it is entirely serious and correct.
Never answer the question you are asked. Rather, answer the question you wish you had been asked.

- Robert S. McNamara - quoted in "Fog of War" -
Posts: 2799
Joined: Sun Jan 15, 2012 1:11 pm
by jbeale » Fri May 25, 2012 8:39 pm
If we have a tested "best recommended practices" way of working from a USB mounted system, it would be a good candidate for some FAQ or howto on the wiki somewhere. Improving the root filesystem speed by a factor of 3x or more is probably something a lot of people would like to do. :-)
User avatar
Posts: 2084
Joined: Tue Nov 22, 2011 11:51 pm
by AndrewS » Fri May 25, 2012 8:41 pm
Joe Schmoe wrote:You don't actually have to write it out to the SD card; you can just mount the image using the loop device and work off of the image.

Yeah, that's what I'd do if I was doing this myself :) ISTR finding that 'kpartx' is a very useful tool for working with the loop device and multiple-partition images such as used here.
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
by AndyOfLinux » Fri May 25, 2012 8:43 pm
Good answers above for the "how"...

I have a 2.5" Crucial m4 60GB SSD connected via a USB hub. R-Pi couldn't quite provide enough power through the built-in USB ports.

Given that the USB drive is way faster as proven by results from the bonnie++ benchmark, I enlarged the swap partition to 1GB - and enabled in /etc/fstab. I upped the vm.swappiness in /etc/sysctl.conf from 1 to 10.

Way faster than running from SDHC :D

- Andy
User avatar
Posts: 7
Joined: Mon Sep 26, 2011 8:54 pm
by rmm200 » Fri May 25, 2012 8:45 pm
Thank you all! Actually the summation of information here beats the sum of the old posts.
Sure gives me something to work on.
Posts: 259
Joined: Sat Mar 03, 2012 10:25 pm
by TonyH » Sat May 26, 2012 11:44 am
I was interested to try this so I have given it a go, followed the great instructions provided here and successfully booted with an SD card and also the (debian image flashed) usb drive plugged in.
This will seem like a silly question I know but how can I tell if it has worked, i.e that my Pi is not just running off the SD card as normal?
Posts: 1
Joined: Sat May 26, 2012 11:34 am
by grumpyoldgit » Sat May 26, 2012 12:21 pm
TonyH wrote:I was interested to try this so I have given it a go, followed the great instructions provided here and successfully booted with an SD card and also the (debian image flashed) usb drive plugged in.
This will seem like a silly question I know but how can I tell if it has worked, i.e that my Pi is not just running off the SD card as normal?


A good test would be to delete the linux partition on the SD card.

This looks like a really good way for me to use up a few 1Gb SD cards I have lying around.
Of course, that means I will have to "loose" a couple of memory sticks, but that's life and I can justify it to myself that my cameras need higher capacity SD cards!

The sun is out and the garden beckons so I will just bookmark this for later!
User avatar
Posts: 1454
Joined: Thu Jan 05, 2012 12:20 pm
by Joe Schmoe » Sat May 26, 2012 3:03 pm
A few comments on the last few responses:
1) If you do it my way (that is, by working with the image rather than a live SD card), there will be no doubt (since the stuff would never be on the SD card at all).
2) You could remove the SD card while the machine is running. If it stays running, you're golden!
Note: You might want to do: umount /boot first.
3) Re: Using a "flash card" (aka, pen drive, aka, etc). In a previous go-around of this thread, I asked whether running it on a flash drive would be workable - i.e., would it be the same as running from a real USB hard drive. The answer that I got was "Maybe, maybe not; you'll have to test and get back to us on that (*).

(*) I'll get back tooya!
Never answer the question you are asked. Rather, answer the question you wish you had been asked.

- Robert S. McNamara - quoted in "Fog of War" -
Posts: 2799
Joined: Sun Jan 15, 2012 1:11 pm
by YorkshireBoy » Sat May 26, 2012 9:23 pm
I have tried this method and it succesfully works. However, inititially I also came across the hang at:

SD 0:0:0:0 [sda] ...

Upon reading the concerns about power from the RPi, I tried aain using a Belkin powered hub. I should state at this point that the USB device I'm using is a 128GB RunCore mini-PCIe SSD, which has an inbuilt USB controller. RunCore don't make this version anymore, but the current ones losted on their site draw 0.5 - 2 W at 3.3V. My rudimentary maths told me I could still be exceeding the USB power limits of the hub.

I then tried a small USB stick. This was rated much lower power draw and worked perfectly. With this in place, I confirmed by cmdline.txt edits were correct.

I then rebooted with the RunCore SSD in the hub, but the USB attached to the RPi. The USB took dev/sda and the RunCore dev/sdb, but powered up correctly and could be seen in GParted. I then used GParted to copy and resize the partition from the USB to the RunCore, removed the USB from the RPi, and hey presto it now boots to the RunCore quite happily.

I assume from this that there is some excess power draw when the RPi initially queries a new drive, and if that power cannot be supplied it will not mount the partition. Once past that point once, it works absolutely fine.
Posts: 2
Joined: Sat May 26, 2012 9:04 pm
by AndrewS » Sat May 26, 2012 10:57 pm
Option 4) Type 'mount' at the command line and see what it lists as the root device.
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
by dukla2000 » Sun May 27, 2012 8:37 pm
Excellent thread this - deserves a bump!

Got the blob on an old 128Mb SD card - finally found a use for it! And Debian on a USB stick. As per other experiences here found the Pi SDcard socket struggling (hdparm -tT or bonnie++) compared to a USB stick. Now can start benchmarking and researching USB sticks.
Posts: 109
Joined: Tue Jan 10, 2012 12:02 am
Location: Reading.UK.EU
by AndrewS » Mon May 28, 2012 10:20 am
Sorry, ignore my option4! Now I have my own Rpi I see it doesn't work :oops:
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
by canyon » Wed Jun 06, 2012 11:29 am
I'm quite proud of myself as I've managed to do this using the following steps on a Windows PC, following a modified version of the steps in the earlier posts:
1. Put the Debian image on to an SD card using Win32DiskImager.
2. Put the Debian image on to a USB stick using Win32DiskImager.
3. With the USB stick selected, use gparted to: Delete the FAT32 partition; enlarge the swap partition to 512M; move the swap partition to the end; enlarge the ext4 partition to fill the space.
4. Working with the FAT32 partition on the SD card in Windows: Download the latest kernel.img and replace the original: edit cmdline.txt to change the root to root=/dev/sda2.

I have one question though:
If I try to boot the Raspberry Pi with a second USB device plugged in when I power up the Raspberry Pi, then I can get a kernel panic, as the kernel reads the second drive as SDA and my rootfs USB stick as SDB and so can't find the root filing system. The workaround here is to only have the USB stick that contains rootfs plugged in when I power up the Raspberry Pi, but it doesn't lead to a 'secure' system.
Is there a different way of specifying the root in cmdline.text using, say, the drive's UUID to make sure the right one is mounted?
Posts: 31
Joined: Sun Jan 29, 2012 9:47 am
by lewmur » Wed Jun 06, 2012 1:03 pm
canyon wrote:I'm quite proud of myself as I've managed to do this using the following steps on a Windows PC, following a modified version of the steps in the earlier posts:
1. Put the Debian image on to an SD card using Win32DiskImager.
2. Put the Debian image on to a USB stick using Win32DiskImager.
3. With the USB stick selected, use gparted to: Delete the FAT32 partition; enlarge the swap partition to 512M; move the swap partition to the end; enlarge the ext4 partition to fill the space.
4. Working with the FAT32 partition on the SD card in Windows: Download the latest kernel.img and replace the original: edit cmdline.txt to change the root to root=/dev/sda2.

I have one question though:
If I try to boot the Raspberry Pi with a second USB device plugged in when I power up the Raspberry Pi, then I can get a kernel panic, as the kernel reads the second drive as SDA and my rootfs USB stick as SDB and so can't find the root filing system. The workaround here is to only have the USB stick that contains rootfs plugged in when I power up the Raspberry Pi, but it doesn't lead to a 'secure' system.
Is there a different way of specifying the root in cmdline.text using, say, the drive's UUID to make sure the right one is mounted?
Haveyou tried swapping the USB sockets the drives are plugged into?
Posts: 284
Joined: Sun Dec 25, 2011 3:20 pm
by hexelpdkk » Wed Jun 06, 2012 1:18 pm
Most references say that you have to have an initrd to do this, or a bootloader that can do it for you. However, I did find this:
http://www.linux-archive.org/gentoo-user/481167-mounting-root-partition-uuid-no-initrd-needed.html
which implies that, with recent kernels, the correct options on the compiled kernel, and a GPT partition table, you might be able to do it.
User avatar
Posts: 128
Joined: Fri Feb 24, 2012 4:40 pm
by canyon » Thu Jun 07, 2012 8:56 am
hexelpdkk - Thanks for the link. When I first read it, I'm afraid it went completely over my head. But, after a lot of thrashing around in Google, I came back to your link and understood it a lot better.
I've still had no success but I think I understand it all a bit more.
All of my efforts fail during boot at :
    sda: sda2: sda3
    sd 0:0:0:0: [sda] Attached SCSI removable disk

I tried :
    root=PARTUUID=<my-uuid> (my-uuid is taken from blkid)
    root=/dev/disk/by-uuid/<my-uuid> (my-uuid is taken from blkid)
    root=/dev/disk/by-id/<my-id> (my-uuid is taken from /dev/disk/by-id)

I'm not suprised that the first two failed as I suspect they need a GPT filesystem, which I don't understand. But I did have hopes of the last one :(

I'm probably missing something simple!
Posts: 31
Joined: Sun Jan 29, 2012 9:47 am
by canyon » Thu Jun 07, 2012 2:46 pm
lewmur - Thanks for the tip.
I've tried all combinations, and at present the 3rd port down on the left side seems to be the most "secure". But when I started, I seem to recall that one failed as well!

Although using the "root=/dev/sda2" format will work if you have no other USB storage devices plugged in at power-on, you can never rely on that.
In my opinion it is a fatal flaw in any PC that it can only find its root filesystem "by chance", so I hope there is a solution that can easily be implemented on the Raspberry Pi.

I've searched around and "root=/dev/disk/by-id/<my-id>" etc. should work, but it seems that they need initramfs in the kernal - This is beyond me :( .

Can someone who knows more about it thow some light on the issue:
Should "root=/dev/disk/by-id/<my-id>" etc. work or not in the current kernel (I'm using rpi-update, so I have the latest)?
If not, is there another solution and how do I do it?
Posts: 31
Joined: Sun Jan 29, 2012 9:47 am
by lewmur » Thu Jun 07, 2012 5:51 pm
canyon wrote:lewmur - Thanks for the tip.
I've tried all combinations, and at present the 3rd port down on the left side seems to be the most "secure". But when I started, I seem to recall that one failed as well!

Although using the "root=/dev/sda2" format will work if you have no other USB storage devices plugged in at power-on, you can never rely on that.
In my opinion it is a fatal flaw in any PC that it can only find its root filesystem "by chance", so I hope there is a solution that can easily be implemented on the Raspberry Pi.

I've searched around and "root=/dev/disk/by-id/<my-id>" etc. should work, but it seems that they need initramfs in the kernal - This is beyond me :( .

Can someone who knows more about it thow some light on the issue:
Should "root=/dev/disk/by-id/<my-id>" etc. work or not in the current kernel (I'm using rpi-update, so I have the latest)?
If not, is there another solution and how do I do it?

I really don't see the problem. So long as your USB drives stay plugged into the same sockets, they should get the same devname each time you boot. So what ever the devname of boot drive is, that's what you should use in the cmdline.txt. If, OTOH, You have a USB drive that is only used occasionally, then just wait until the system finishes booting before plugging it in.
Posts: 284
Joined: Sun Dec 25, 2011 3:20 pm
by AndrewS » Sat Jun 09, 2012 2:31 pm
This is a problem I've seen before (but not needed to fix myself) - have a look at this discussion, especially the last post in the thread https://groups.google.com/group/bifferb ... 517d0aeda7
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
by Joe Schmoe » Sat Jun 09, 2012 4:18 pm
AndrewS wrote:This is a problem I've seen before (but not needed to fix myself) - have a look at this discussion, especially the last post in the thread https://groups.google.com/group/bifferb ... 517d0aeda7


This is interesting. For reference, here is that last post:

--- Cut ---
The way I have got round this previously (as I quite frequently run
multiple USB disks connected to the BB) is to disable Asynchronous
SCSI Scanning in the kernel. This does require you to essentially roll-
you-own, but means that when the mass storage devices are detected, it
will go through them in the order on the USB bus, rather than
whichever finishes first. With this disabled, you should find that /
dev/sda is always the same device etc.
--- Cut ---

Does this mean that in the default configuration (i.e., with "Asynchronous
SCSI Scanning" on), it really is *random* (i.e., random which device turns up as /dev/sda and which one turns up as /dev/sdb, etc) ? And by "random", I mean it can vary from boot to boot without making any hardware changes (i.e., without unplugging or plugging anything).

This is strange to me, because I always assumed that whatever it was, it would be the same (as long as you didn't change anything). I.e., that the problem being discussed in the present thread only arise when you do plugging and/or unplugging between boots. Is it really that way (i.e., "random") ?
Never answer the question you are asked. Rather, answer the question you wish you had been asked.

- Robert S. McNamara - quoted in "Fog of War" -
Posts: 2799
Joined: Sun Jan 15, 2012 1:11 pm
by Lob0426 » Mon Jun 11, 2012 9:10 pm
The USB HDD power problem is solved by using a "Y" cable to the drive. It steals power from two ports. I am using a Startec SATA cord that is only about a foot long. No problems on the Panda Board through a powered hub at all. I will try it on the RasPi here shortly. Good info in this thread.

Can you identify the drive by UUID or Label instead dev? I used UUID on the Panda Board. It did not care where in the hub I plugged it in. Ubuntu also lets you use the label, but I do not know if Debian does.
512MB version 2.0 as WordPress Server
Motorola Lapdock with 512MB
Modded Rev 1.0 with pin headers at USB

http://rich1.dyndns.tv/
(RS)Allied ships old stock to reward its Customers for long wait!
User avatar
Posts: 1942
Joined: Fri Aug 05, 2011 4:30 pm
Location: Susanville CA.
by AndrewS » Tue Jun 12, 2012 1:07 am
Lob0426 wrote:Ubuntu also lets you use the label, but I do not know if Debian does.

Ubuntu (on a PC) uses Grub, which is much more sophisticated than the bootloader used on the RaspberryPi...
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK