Using usb hard drive as default boot and storage?


28 posts   Page 1 of 2   1, 2
by alltronics » Fri Jun 22, 2012 7:21 pm
Hi,

I'm sure this must have been asked, but can't find the topic. How can I use my usb hard drive (750GB) as the default boot drive and storage area? I only have a 4GB SD and not much room for software. I'd like to use Moovida or similar media centre and run it from the hard disk. Is this possible??

Thanks for any help.

Theo
Posts: 7
Joined: Wed May 30, 2012 7:14 pm
by jecxjo » Fri Jun 22, 2012 8:10 pm
There is a fairly simple layout for doing this...

The Pi's GPU startup code looks for the first partition on an SD Card (formatted as Fat32) and loads the bootcode into RAM. This bootcode then in turn loads the kernel from the same partition and eventually you get to your Root File System on the second partition. What you'll want to do is modify your kernel arguments to point to your USB drive instead of your SD card when it comes to the RFS. Make a partition on the HDD, copy files, etc and there you go. (Sorry Its a little bit more cumbersome than first stated).

I'm currently writing up this process and I'll post the link once I'm done.
IRC (w/ SSL Support): jecxjo.mdns.org
Blog: http://jecxjo.motd.org/code
ProjectEuler Friend: 79556084281370_44d12dd95e92b1d9453aba2bdc94101b
User avatar
Posts: 157
Joined: Sat May 19, 2012 5:22 pm
Location: Ames, IA (USA)
by alltronics » Fri Jun 22, 2012 8:13 pm
Thanks, a step by step guide would be great.

Theo
Posts: 7
Joined: Wed May 30, 2012 7:14 pm
by alltronics » Sat Jun 23, 2012 3:43 pm
I found this:
http://forum.stmlabs.com/showthread.php?tid=481

so when the raspbmc downloads are up again I'll try it with that.

I tried to install moovida in the standard debian and and it stopped my R-Pi from booting... so that doesn't work! There were a lot of missing packages and moovida is probably too heavy for it anyway.

Thanks,

Theo
Posts: 7
Joined: Wed May 30, 2012 7:14 pm
by andreword » Tue Jun 26, 2012 4:39 pm
Not sure if this will help you but I was having a lot of problems with my SD cards so I set about setting up a clean debian install on an external HDD, and using a smaller more reliable SD card for the boot files

I've subsequently tried this with Raspmbc and other setups to try and provide additional swap space without wearing out the SD card (by copying the ext2 file system from the SD card to the external USB drive and modifying the boot command line to match)

Link to my instructions : http://andreword.co.uk/Applications/Raspberry_PI/Entries/2012/6/4_Preparing_the_destination_drive.html

I've extracted the full instructions below from my site
Using the most recent Debian SD card image, I’ve booted up with the external drive attached to the second USB port

1.Logged in to debian with the pi/raspberry account
2.sudo su (switch to su mode without the root password)
3.ls /dev
(list all devices, depending on the type of drive)
Look for /dev/sda
Note your SD card will be listed as /dev/mmcblk0 (with partitions p1, p2, p3)
4.fdisk /dev/sda (obviously change this to match your HDD configuration)
5.p (print the partition table)
6.d <Enter> <number> (for each partition to delete them) (Note this will wipe your drive, do not proceed unless the attached USB HDD drive is an empty drive)
7.n <Enter> p <Enter> 1 <Enter> (Choose appropriate size for your main file area)
8.n <Enter> p <Enter> 2 <Enter> (make sure to have left yourself some swapspace)
9.You can set the boot flag if you want, but it isn’t needed since your booting from the SD card
10.mkfs.ext4 /dev/sda1 (format the first partition)
11.mkswap /dev/sda2 (format the swap space)

Now we need to prepare the installation onto this new drive ready to boot
1.mkdir /a (create a folder called a in the root, this is where we will build our new install)
2.mount /dev/sda1 /a (mount our new drive on /a)
3.apt-get update (update our package sources)
4.apt-get install debootstrap (install debootstrap)
5.debootstrap squeeze /a http://192.168.1.11:3142/ftp.debian.org/debian/
(do a minimal build onto our folder)
I use apt-cacher-ng on another machine, use http://ftp.debian.org/debian/

Once the packages have downloaded and installed we should have an extremely minimal debian install in /a
Let’s move across into this new environment and start to build it up to be able to operate on it’s own

Download the kernel file from http://www.bootc.net/raspberrypi/linux- ... _armel.deb
(Note : You may find that the version has been updated since I wrote this note, please go to http://www.bootc.net/raspberrypi/ to find the latest file name if needed)

1. wget http://www.bootc.net/raspberrypi/linux- ... _armel.deb
Copy the downloaded files, linux-image-3.2.18-rpi1+_5_armel.deb and linux-raspberrypi-3.1.9-18-arm.pkg.tar to your new area (/a/root/installation was where I put the files)

1.chroot /a (Change root to our new folder)
2.passwd <Enter> (follow instructions to set the new passwd for root on your new install)
3.apt-get update
4.apt-get install aptitude openssh-server locales ntp
5.dpkg-reconfigure locales

This should provide us with a remotely accessible platform once we get the kernel installed

Let’s get the kernel and modules installed
1.cd /root/installations
2.dpkg -i linux-image-3.2.18-rpi1+_5_armel.deb

The new kernel is now stored in /boot. In order to update your SD card to use this kernel you now need to do a little slight of hand

The file you need is in /boot called vmlinuz-3.2.18-rpi1+, and to use it we need to copy this file and rename it to kernel.img and store it on the SD card

When I applied the kernel update on my SD card, I did the following
1.mkdir /tempSD
2.mount /dev/mmcblk0p1 /tempSD
3.mv /tempSD/kernel.img /tempSD/kernel.img.original
4.cp /boot/vmlinuz-3.2.18-rpi1+ /tempSD/kernel.img
5.nano /tempSD/cmdline.txt
6.(modify the boot command by changing root=/dev/mmcblk0p2 to root=/dev/sda1)
7.CTRL+O,<Enter>,CTRL+X
8.umount /tempSD
9.rmdir /tempSD

These changes now mean you have the up to date kernel on the SD card and you’ve modified the boot commands to use your USB HDD as the root file system

Now do a reboot (reboot <Enter>) and let’s see what happens

The first thing you should notice is that it boots up fairly quickly. Always a good sign I think

The second is that you should be able to logon as root, using the password you set earlier

Now we need to make a few modifications to the configuration in order to automatically get the internet connection up and running

1.Log on to the PI
2.nano /etc/network/interfaces
3.Add the following lines to the file if they are not already present
auto eth0
iface eth0 inet dhcp
4.Before we forget, we need to turn on our swap if it hasn’t already
5.nano /etc/fstab
add the following line
none /dev/sda2 swap sw 0 0
6.exit
7.swapon /dev/sda2
8.To address a bug I keep finding with the network connection, edit the sysctl.conf file in /etc and include the line vm.min_free_kbytes=32768
9.Now you should be able to reboot and have networking available
Posts: 2
Joined: Tue Jun 26, 2012 4:26 pm
by jgearing » Sun Jul 01, 2012 8:10 am
Andreword
I have been trying out your instructions for using my usb hdd
My first problem comes with the line
11.mkswap /dev/sda2
this tells me the file does not exist although is shows up in fdisk
I decided to carry on regardless and then when i reached line 5
debootstrap squeeze etc, it was unable to download the file.
I followed the link and the page appears to be dead
Any suggestions
I am using wheezy if that makes any difference, I cannot get the LAN working with a powered hub
in squeeze

thanks

JG
Posts: 51
Joined: Sat Mar 03, 2012 6:33 pm
by mxcum167 » Sun Jul 01, 2012 9:01 am
Here is how I did it.

Boot from my SD card (Debian), with USB Drive connected.

Start up X windows as root (sudo startx)

Run gparted, and create partition on USB drive (use ext4 format, I will assume it is called /dev/sda1)
(you may need to install gparted (apt-get install gparted)

mount the USB drive partition
sudo mkdir /mnt/newroot
sudo mount /dev/sda1 /mnt/newroot

Copy contents of / (root partition to USB /dev/sda1)
(You may need to install rsync, (apt-get install rsync)
sudo rsync -avxS / /mnt/newroot

Change /boot/cmdline.txt
root=/dev/sda1

Reboot RPi.
Confirm you're rootfs is now on /dev/sda1 (The size of the partition will be a big give away here.)

You can now delete the old partition on the SD Card if you want to. I have kept mine as an emergency rootfs if I need it.


I think that is all, this is from memory as I am not near my RPi at the moment, but I have done this about 6 times now on two different RPi's, and I have not had a problem yet. :)
Martin www.chez-cummings.com
User avatar
Posts: 41
Joined: Sun Mar 04, 2012 11:56 am
Location: UK
by jgearing » Sun Jul 01, 2012 11:26 am
not much luck there
GParted couldn't see my HDD
I kept refreshing and it appeared for about 5 secs and then vanished
may be cheapo power hub but it was OK when I tried the other method

thanks anyway

JG
Posts: 51
Joined: Sat Mar 03, 2012 6:33 pm
by Joe Schmoe » Sun Jul 01, 2012 12:44 pm
Just to be sure - you do know about "extending your partition", right?

The reason I ask is that "space" is usually not the reason why people ask about booting from USB, vs booting from the SD card. Speed and/or reliability usually is.

The point is that assuming you have the usual 16GB or larger (or even 8GB for that matter) SD card, then if you can arrange to be able to use it all, then you should be OK on the "space" frontier. There are about 1,000,000,000 posts on here about how to do the "extend" operation.
Never answer the question you are asked. Rather, answer the question you wish you had been asked.

- Robert S. McNamara - quoted in "Fog of War" -
Posts: 2256
Joined: Sun Jan 15, 2012 1:11 pm
by Nr90 » Sun Jul 01, 2012 12:47 pm
You could write the image to both your sd card and your HDD.
Then remove the first partition on the HDD (the fat32 one).
Change the /boot/cmdline.txt to point it to /dev/sda1 (on the sd card - fat32 partition).
Expand the ext4 partition on the HDD to your preferred size and you should be good to go.
You can either remove the ext partition from the SD card or leave it there as a backup.
Same result as posted above, however slightly easier to do IMO.
Posts: 213
Joined: Sat Nov 26, 2011 12:39 pm
by recantha » Sun Jul 01, 2012 5:22 pm
I Nr90 gives a very easy-to-do though not frustration-free method.

I tried it. It really does work. Once I get the whole thing done, I'll post my method.
However, if you are going to do this, make sure that you expand the partition on the USB drive BEFORE you reboot. ;-) Because you can't then expand it once you've booted from it.

Of course, keeping the ext partition on the SD card as a 'backup' is okay... but of course if you need to use it you have the problem that the config still points at the now-knackered USB drive.

One question, Nr90... if I wanted to change the config back to use the ext partition on the SD card, where would I find the cmdline.txt on the card? I know, I know, probably a simple question. It has been... *cough cough* years since I used Linux or Unix at all. ;-)

Thanks for the suggestion tho!
My Raspberry Pi blog with all my latest projects and links to articles
http://raspberrypipod.blogspot.com. +++ Current project: PiPodTricorder - lots of sensors, lots of mini-displays, breadboarding, bit of programming.
Posts: 209
Joined: Mon Jun 25, 2012 10:41 am
by mxcum167 » Sun Jul 01, 2012 9:44 pm
recantha wrote:.... if I wanted to change the config back to use the ext partition on the SD card, where would I find the cmdline.txt on the card? I know, I know, probably a simple question. It has been... *cough cough* years since I used Linux or Unix at all. ;-)


cmdline.txt is always in the /boot partition. It is also the only partition visible when the SD Card is put into a PC/MAC.
Martin www.chez-cummings.com
User avatar
Posts: 41
Joined: Sun Mar 04, 2012 11:56 am
Location: UK
by recantha » Thu Jul 05, 2012 7:02 am
@andreword

Thanks for the instructions. Your method works, albeit with some tweaks. I'm writing up the changes and will post here when I'm done.

Could you tell me what powered hub you're using? I've been a bit frustrated trying to find one that definitely works and yours sounds stable!

Thanks,
--
Mike
My Raspberry Pi blog with all my latest projects and links to articles
http://raspberrypipod.blogspot.com. +++ Current project: PiPodTricorder - lots of sensors, lots of mini-displays, breadboarding, bit of programming.
Posts: 209
Joined: Mon Jun 25, 2012 10:41 am
by gemani » Sat Jul 14, 2012 11:23 am
I've been trying to get this working according to mxcum167's suggestion, (you need to specify '--exclude newroot' to rsync though or it recurses!). I have the entire root fs on the drive as far as I can tell but the boot just hangs, presumably when it's trying to find the root filesystem. Has the drive been mounted as /dev/sda1 at that point? Also, using that method there doesn't appear to be provision for a swap partition?
Posts: 1
Joined: Sat Jul 14, 2012 10:57 am
by Thorprime » Sun Jul 15, 2012 4:27 am
gemani wrote:I've been trying to get this working according to mxcum167's suggestion, (you need to specify '--exclude newroot' to rsync though or it recurses!). I have the entire root fs on the drive as far as I can tell but the boot just hangs, presumably when it's trying to find the root filesystem. Has the drive been mounted as /dev/sda1 at that point? Also, using that method there doesn't appear to be provision for a swap partition?


I have a similar problem.
I have formated the USB drive according to this spec: http://forum.stmlabs.com/showthread.php?tid=481
I can mount /dev/sda2 when booting off the SD card and see the root filesystem there. However, when I reboot with root=/dev/sda2, the pi doesn't boot. I see "Waiting for root device /dev/sda2..." in the boot sequence. I see the sd card listed with mmcblk0: p1 p2 p3
I also see usb 1-1.2: Product: Simple Drive (with many other usb 1-1.2 lines) followed by scsi0 : usb-storage 1-1.2:1.0

The last line printed is:
scsi 0:0:0:0 Direct-Access STECH Simple Drive 8.59 PQ: 0 ANSI: 0

It looks like it has found the drive but failed to map it to /dev/sda .

Is there something I could be missing?
Posts: 1
Joined: Thu Jun 21, 2012 10:37 pm
by andreword » Sun Jul 15, 2012 12:38 pm
recantha wrote:@andreword

Could you tell me what powered hub you're using? I've been a bit frustrated trying to find one that definitely works and yours sounds stable!


I'm using a cheap 4 port powered hub from Logik I found on offer in Currys or somewhere. The only stability issues I've had have been around SD card compatibility, and more recently USB HID device issues. (keys repeating until another key is pressed)
Posts: 2
Joined: Tue Jun 26, 2012 4:26 pm
by gazmandev » Fri Jul 27, 2012 11:21 am
The last line printed is:
scsi 0:0:0:0 Direct-Access STECH Simple Drive 8.59 PQ: 0 ANSI: 0


I get the same error using the Debian/Raspbian release you can order pre-imaged from the suppliers trying to use a USB HDD (old-skool with spinning platters). I have some swap space on this drive, no problems.

Have also tried editing the /etc/fstab and telling it to mount / on /dev/sda1 (I took a backup first, just in case!). While this appears to work it actually doesn't. You end up with the system thinking the rootfs is on the external hdd, but the amount of free space reported is still the amount of free space on the SD card.

Pretty annoying, especially as I really don't want to have to go through the entire process of recompiling a kernel in the off-chance it may work after hours spent downloading tool chains, source code, compiling kernels etc. Wonder if it's related to the particular make/model of USB HDD... :?

Anyway, resorting to trying ArchLinux as it appears Fedora is no longer a supported release. Will see how that handles it.
Posts: 12
Joined: Wed Jul 11, 2012 10:12 am
by VanillaMoz » Wed Aug 08, 2012 6:01 am
Hi, how many TB external hard disk available in market today ?
Posts: 1
Joined: Wed Aug 08, 2012 5:59 am
by ramtripper » Tue Aug 28, 2012 2:07 pm
does this mean it's now possible to boot windows?

windows 8 has the "windows to go" feature and can be installed onto a usb drive. It runs fully and saves every session so it's fully operational.

not a big fan of windows, but this opens new possibilities if it does work.
Posts: 1
Joined: Tue Aug 28, 2012 2:04 pm
by itimpi » Tue Aug 28, 2012 3:32 pm
ramtripper wrote:does this mean it's now possible to boot windows?

windows 8 has the "windows to go" feature and can be installed onto a usb drive. It runs fully and saves every session so it's fully operational.

not a big fan of windows, but this opens new possibilities if it does work.

No. Windows 8 is not compatible with the ARM processor used by the Pi.
Posts: 1027
Joined: Sun Sep 25, 2011 11:44 am
Location: Potters Bar, United Kingdom
by egp » Fri Sep 14, 2012 7:38 pm
I've been trying to get this to work for a while, without success. I tried following the procedure above, using wheezy instead of squeeze. My current failure is that when I try to
Code: Select all
chroot
, I get a segmentation fault.

I assume numerous people have this working - would someone please update the instructions?
Posts: 5
Joined: Thu Dec 29, 2011 5:52 pm
by alvarogmj » Sun Nov 25, 2012 11:31 pm
gazmandev wrote:
The last line printed is:
scsi 0:0:0:0 Direct-Access STECH Simple Drive 8.59 PQ: 0 ANSI: 0


I get the same error using the Debian/Raspbian release you can order pre-imaged from the suppliers trying to use a USB HDD (old-skool with spinning platters). I have some swap space on this drive, no problems.

Have also tried editing the /etc/fstab and telling it to mount / on /dev/sda1 (I took a backup first, just in case!). While this appears to work it actually doesn't. You end up with the system thinking the rootfs is on the external hdd, but the amount of free space reported is still the amount of free space on the SD card.

Pretty annoying, especially as I really don't want to have to go through the entire process of recompiling a kernel in the off-chance it may work after hours spent downloading tool chains, source code, compiling kernels etc. Wonder if it's related to the particular make/model of USB HDD... :?

Anyway, resorting to trying ArchLinux as it appears Fedora is no longer a supported release. Will see how that handles it.

I'm trying the same thing, and getting the same problem. It seems the system is not able to recognize the partitions. Is there anything we can try in order to troubleshoot? I'm using SlackwareARM as OS.
Posts: 3
Joined: Sun Nov 25, 2012 11:27 pm
by itimpi » Sun Nov 25, 2012 11:54 pm
You specify in the cmdline.txt file on the SD card where the root filing system is to be located. I use that to run off a USB hard disk on one of my Pi's.
Posts: 1027
Joined: Sun Sep 25, 2011 11:44 am
Location: Potters Bar, United Kingdom
by alvarogmj » Mon Nov 26, 2012 12:07 am
itimpi wrote:You specify in the cmdline.txt file on the SD card where the root filing system is to be located. I use that to run off a USB hard disk on one of my Pi's.

Yup, that is done (by the SlackwareARM installer, btw). This is my cmdline.txt
Code: Select all
dwc_otg.lpm_enable=0 console=tty1 root=/dev/sda1 rootfstype=xfs rootwait ro

I get a "waiting for /dev/sda1" message and then the boot process is stopped, just as described by post viewtopic.php?p=123480#p123480
Posts: 3
Joined: Sun Nov 25, 2012 11:27 pm
by M5 » Tue Dec 04, 2012 4:34 pm
I’m new to Linux but I followed mxcum167’s instructions and it works really well. Thanks.
I then moved on to create a swap partition and have managed to get that up and running using the information in this thread Thanks all. But, yes there’s always a but. Can anyone tell me how to stop dphys-swapfile from loading?
User avatar
Posts: 4
Joined: Tue Mar 27, 2012 2:35 pm