mgherghina
Posts: 4
Joined: Sat Jun 25, 2016 8:41 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Jun 26, 2016 4:37 am

yes but it won't do it in f2fs. Right?

SonOfAMotherlessGoat
Posts: 690
Joined: Tue Jun 16, 2015 6:01 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Jun 26, 2016 4:44 am

Shoot 4Dee a message and ask for that feature. Dan's really accommodating for his users.
Account Inactive


malvcr
Posts: 50
Joined: Mon Apr 13, 2015 3:31 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Mon Jul 11, 2016 10:31 pm

This is a very busy discussion thread; it seems that it is really important to have something different than the SD card with the operating system inside.

In fact, this is something I have been fighting for several months on RPI2 and RPI3 machines with different grades of success. For me, the main issue when doing it on a hard disk is about the time "before" booting and "after" shutdown, being this last one the most critical part.

When following the instructions, the Raspberry is able to wait for the root partition to be ready. BUT, when shutting down, the machine quits the electricity on the hard disk before the disk finish closing everything. In this case, the 100% of the times I correctly shutdown the machine, it needs to be recovered on the next startup and this is basically an unacceptable behavior if I like to keep my data safe. I didn't find any parameter or trick helping me on the Raspberry side.

So, I came with a natural solution that in fact works 100% of the times. I don't connect the hard disk directly to the Raspberry, but on a powered USB hub. Then the Raspberry has two cables to the USB hub, for power and data, and the hard disk one for data (it seems this won't work on model A because of backfeeding).

In this case, when the Raspberry shuts down and quits the electricity to the USB ports, the hard disk still receives electricity from the powered USB hub, having the capacity to complete the hardware shutdown counterpart. Later, when I turn on the Raspberry, the hard disk is working as a charm with zero corruption errors.

Edit:

In fact I have some errors, because although the hard disk is behaving well, I have a beautiful colored square on the upper right size of the screen each time I perform some task as a "ls" or "pwd" operation. This means that my USB HUB is not providing enough power to the Rpi3.

According with http://raspi.tv/2016/how-much-power-doe ... ed-to-pi2b, I need around 720 milliamps to work well with four cores and my USB 2.0 only will provide 500 mA as maximum. Here I have two options that I will try tomorrow (1) to turn off three cores and (2) to use a 900 mA USB 3.0 powered hub. In this case, the first option doesn't have a lot of sense .. in that case it is better to use a Rpi Zero than a Rpi 3.

katieswift
Posts: 2
Joined: Tue Jul 12, 2016 7:53 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jul 14, 2016 10:28 am

you can follow these steps:
1. Do a fresh install and configure your Raspi making sure not to expand the filesystem or update.
2. Install some tools we’ll need later.
sudo apt-get update
sudo apt-get install rsync gdisk
3. Check to see if usb is recognized and check the partitions.
lsusb
sudo lsblk
pi@raspberrypi ~ $ lsusb
Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
Bus 001 Device 004: ID 0930:6545 Toshiba Corp. Kingston DataTraveler 102 Flash Drive
pi@raspberrypi ~ $ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 14.6G 0 disk
└─sda1 8:1 1 14.6G 0 part
mmcblk0 179:0 0 3.8G 0 disk
├─mmcblk0p1 179:1 0 56M 0 part /boot
└─mmcblk0p2 179:2 0 2.7G 0 part /
As you can from lsusb the USB stick is recognized on device 004 and lsblk tells us that the USB stick is on sda. From this point on I’ll use /dev/sda for the USB drive.

4. Start up gdisk and setup partition table.
sudo gdisk /dev/sda
pi@raspberrypi ~ $ sudo gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.5

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help):
4.1 Use p to list the current partitions if there are any.
Command (? for help): p
Disk /dev/sda: 30489408 sectors, 14.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): FF07443F-D359-4DAF-9399-BFCD862CF37F
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 30489374
Partitions will be aligned on 2048-sector boundaries
Total free space is 2813 sectors (1.4 MiB)

Number Start (sector) End (sector) Size Code Name
1 2048 30488575 14.5 GiB 0700
4.2 Delete old partition(s) with d
4.3 Make your new partition with n , when prompted to enter information hit Enter to use defaults.
Command (? for help): n
Partition number (1-128, default 1):
First sector (34-30489374, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-30489374, default = 30489374) or {+-}size{KMGTP}:
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'
4.4 Get the partition information with i and save your Partition unique GUID to a text file temporarily.
Command (? for help): i
Using 1
Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem)
Partition unique GUID: FBEC95EC-C8A6-4F66-9BA3-0661B1359D0A
First sector: 2048 (at 1024.0 KiB)
Last sector: 30489374 (at 14.5 GiB)
Partition size: 30487327 sectors (14.5 GiB)
Attribute flags: 0000000000000000
Partition name: 'Linux filesystem'
The partition GUID is highlighted in bold.

4.5 Write the changes with w
Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.
5. Format the partition and mount it
sudo mke2fs -t ext4 -L rootfs /dev/sda1
sudo mount /dev/sda1 /mnt
6. Copy the filesystem over to your new partition with rsync
sudo rsync -axv / /mnt
This is going to take 15-25 minutes depending on your sd card and USB speeds so take this chance to go get yourself a drink.

7. Backup your cmdline.txt file and edit it to include your partition.
sudo cp /boot/cmdline.txt /boot/cmdline.backup
sudo nano /boot/cmdline.txt
7.1 Edit the file and change your “root=” to point to your new filesystem partition.
dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=PARTUUID=FBEC95EC-C8A6-4F66-9BA3-0661B1359D0A rootfstype=ext4 elevator=deadline rootwait
Use the unique GUID you saved earlier.

8. Use blkid to get your partition UUID for fstab
pi@raspberrypi ~ $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" LABEL="boot" UUID="993B-8922" TYPE="vfat"
/dev/mmcblk0p2: UUID="fc254b57-8fff-4f96-9609-ea202d871acf" TYPE="ext4"
/dev/sda1: LABEL="rootfs" UUID="bd9300dd-1d44-4a1b-8316-fbd5407fd9f7" TYPE="ext4"
9. Open the fstab file on your new filesystem
sudo nano /mnt/etc/fstab
proc /proc proc defaults 0 0
/dev/mmcblk0p1 /boot vfat defaults 0 2
#/dev/mmcblk0p2 / ext4 defaults,noatime 0 1
/dev/disk/by-uuid/YOUR-UUID / ext4 defaults,noatime 0 1

# a swapfile is not a swap partition, so no using swapon|off from here on, use dphys-swapfile swap[on|off] for that
Comment out the old filesystem at /dev/mmcblk0p2 and add the new line pointing to your new filesystem to the bottom of your file and add your UUID where indicated.

10. Reboot
If all went well your Raspberry Pi should be up and running using your new filesystem on your USB stick. To double check that you’re running off the USB stick use the command lsblk .

pi@raspberrypi ~ $ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 14.6G 0 disk
└─sda1 8:1 1 14.6G 0 part /
mmcblk0 179:0 0 3.8G 0 disk
├─mmcblk0p1 179:1 0 56M 0 part /boot
└─mmcblk0p2 179:2 0 2.7G 0 part

gkreidl
Posts: 5245
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jul 14, 2016 1:49 pm

@ katieswift:I didn't read everything but you start with contradictions:
1) Don't expand the file system and don't update: the file system is now expanded automatically
2) Then you run update and install, in spite of what you said before,

Everything you need can be found in the OP. Did you read it before posting?
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

sineverba
Posts: 11
Joined: Tue Jul 16, 2013 3:27 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sat Jul 16, 2016 9:01 am

Hi to all!
Using from several years this method and it works like a charm (SD for first boot + USB pen drive mounted as / ).

cmdline.txt in SD card has

Code: Select all

root=/dev/sda1
/etc/fstab (on the USB stick) has the following entries AND sdb1 is an error, but it works!!! (It's my error because I tried to change USB - see next doubts! - without success), but it works. SDB1 doesn't exists on the system, but it works (why?)

Code: Select all

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
#/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
#/dev/sda1      /       ext4    defaults,noatime        0       1
/dev/sdb1       /       ext4    defaults,noatime        0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
Update: removed sdb1, reinserted sda1 and Raspi start without problem. Is it normal? :shock:



My doubts are following:

1) Let's say I would make a 1:1 copy, to install another raspberry on another location, without reinstall software, tuning, etc.

SD stuff:
I made an image of the SD via Win32DiskImager saved on another PC.
USB stuff:
I'm using

Code: Select all

rsync -xav / /media
to make a 1:1 update copy of the stick on another stick (mounted on

Code: Select all

/media
of course)

I'm not using the UUID system, but mounted as /dev/sdax on fstab.

What I need to do to render perfectly functioning the copy on another Raspberry?

2) Let's say the USB dead. I have the other stick (see 1) with same file.

What I need to do to render perfectly functioning the other USB copy on same Raspberry?

3) Let's say the SD card dead. => Simple, I will use the image archived...

I hope I'm clear. Thank you very much!

sagdih
Posts: 2
Joined: Tue Jul 26, 2016 4:06 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Jul 26, 2016 4:10 pm

I am new to Raspberry PI's and found this article very helpful. One question, once I get my OS installed on my HDD what is the proper method to make a new SD card lets say a small 2 Gig card that just has my /boot on it? Everything I have tried results in a PI that wont boot.

sagdih
Posts: 2
Joined: Tue Jul 26, 2016 4:06 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Jul 27, 2016 10:29 pm

Well I figured it out, putting a link here just in case anyone else is looking for the info.
http://sysmatt.blogspot.com/2014/08/bac ... -your.html

California
Posts: 149
Joined: Sat Jun 06, 2015 2:33 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jul 29, 2016 3:25 pm

Thanks to katieswift, very clear mannual step by step. Works 100%.

Regards,

Vlado

Ruffio
Posts: 16
Joined: Fri Jun 17, 2016 2:35 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Mon Aug 08, 2016 8:44 am

There have now been published this 'official' procedure: https://github.com/raspberrypi/document ... des/msd.md

It is still in beta, but try it out. The more people that tries it out and report potential bugs, the faster a stable official process will be developed.

sineverba
Posts: 11
Joined: Tue Jul 16, 2013 3:27 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Mon Aug 08, 2016 8:49 am

I will try with the 2... or someone know that for sure it doesn't function?

Edit: I did try with the PI2. All instructions goes well, but PI doesn't restart. Sigh.

pgloor
Posts: 8
Joined: Sat Aug 08, 2015 5:20 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Sep 11, 2016 1:48 pm

sineverba wrote:I will try with the 2... or someone know that for sure it doesn't function?

Edit: I did try with the PI2. All instructions goes well, but PI doesn't restart. Sigh.
The boot code is stored in the BCM2837 device only, so the Pi 1, Pi 2, and Pi Zero will all require SD cards.
See https://www.raspberrypi.org/blog/pi-3-b ... rage-boot/ for detailed information..

sudhir_brahma
Posts: 22
Joined: Fri Jun 19, 2015 12:08 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Mon Oct 03, 2016 1:01 pm

Very useful! Pl keep this link active.

ravustaja
Posts: 8
Joined: Thu Oct 13, 2016 11:06 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Oct 13, 2016 11:19 am

I used the 1st method before, but now I've realized I could actually use another drive sticked into my Rpi.
I tried changing the dev/sda2 part in my cmdline.txt to PARTUUID (I tried both, the longer one (sudo gdisk) and the shorter one (sudo blkid)). I also changed the line from fstab. It wouldn't reboot after that, but I had backups to fix that issue.

My question is, that is is possible to change to the 2nd way, as I have already done the partitioning/formatting the easier way? I have installed a lot of packages and done other stuff to the OS, so I'd rather not go through the hassle of starting all over again.

I also noticed that gdisk says "Found invalid GPT and valid MBR" about the partitions, does that have anything to do with why it's not booting?

Thanks

User avatar
bleep42
Posts: 129
Joined: Wed Mar 07, 2012 12:43 pm
Location: Sussex
Contact: Website

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Oct 13, 2016 4:38 pm

ravustaja wrote:I used the 1st method before, but now I've realized I could actually use another drive sticked into my Rpi.
I tried changing the dev/sda2 part in my cmdline.txt to PARTUUID (I tried both, the longer one (sudo gdisk) and the shorter one (sudo blkid)). I also changed the line from fstab. It wouldn't reboot after that, but I had backups to fix that issue.

My question is, that is is possible to change to the 2nd way, as I have already done the partitioning/formatting the easier way? I have installed a lot of packages and done other stuff to the OS, so I'd rather not go through the hassle of starting all over again.

I also noticed that gdisk says "Found invalid GPT and valid MBR" about the partitions, does that have anything to do with why it's not booting?

Thanks
Hi,
I've just done what you are trying to do to 2 pies, because I was was having boot up problems since updating to the latest raspbian, both worked fine, still are. I used the ids from the blkid command, which strangely seem to be different to the ones using the gdisk command, yes I also had the "Found invalid GPT and valid MBR" error, so that is not your problem.
Make sure you are using the correct UUID= and PARTUUID= both from blkid, mine are listed on the same line as the /dev/sda1:
So in my cmdline.txt I have
root=PARTUUID=000c66b8-01
this file must be only 1 line of text, so everything goes on a single line.
In fstab I have
/dev/disk/by-uuid/2f917d2e-dfea-4e68-8d0f-6387b3e6bf0e / ext4 defaults,noatime 0 1
Obviously your ids will be different.
Good luck. Kevin.

ravustaja
Posts: 8
Joined: Thu Oct 13, 2016 11:06 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Oct 20, 2016 9:34 am

bleep42 wrote:
ravustaja wrote:I used the 1st method before, but now I've realized I could actually use another drive sticked into my Rpi.
I tried changing the dev/sda2 part in my cmdline.txt to PARTUUID (I tried both, the longer one (sudo gdisk) and the shorter one (sudo blkid)). I also changed the line from fstab. It wouldn't reboot after that, but I had backups to fix that issue.

My question is, that is is possible to change to the 2nd way, as I have already done the partitioning/formatting the easier way? I have installed a lot of packages and done other stuff to the OS, so I'd rather not go through the hassle of starting all over again.

I also noticed that gdisk says "Found invalid GPT and valid MBR" about the partitions, does that have anything to do with why it's not booting?

Thanks
Hi,
I've just done what you are trying to do to 2 pies, because I was was having boot up problems since updating to the latest raspbian, both worked fine, still are. I used the ids from the blkid command, which strangely seem to be different to the ones using the gdisk command, yes I also had the "Found invalid GPT and valid MBR" error, so that is not your problem.
Make sure you are using the correct UUID= and PARTUUID= both from blkid, mine are listed on the same line as the /dev/sda1:
So in my cmdline.txt I have
root=PARTUUID=000c66b8-01
this file must be only 1 line of text, so everything goes on a single line.
In fstab I have
/dev/disk/by-uuid/2f917d2e-dfea-4e68-8d0f-6387b3e6bf0e / ext4 defaults,noatime 0 1
Obviously your ids will be different.
Good luck. Kevin.
I managed to get it working. Thanks! :)

anderml1955
Posts: 5
Joined: Tue Nov 22, 2016 2:55 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Nov 22, 2016 3:06 am

I've had a RPI3 booting & running from a USB drive for a few weeks. I followed the procedure on https://github.com/raspberrypi/document ... des/msd.md and it works great.

I use Webmin and just this last week noticed several updates - raspberrypi-bootloader armhf Raspberry Pi bootloader New version 1.20161116-1 Jessie & raspberrypi-kernel armhf Raspberry Pi bootloader New version 1.20161116-1 Jessie.

I suspected updating the system with those would break the boot process so I cloned the USB drive and ran the update. As expected the drive does not boot. I'm now running on my original USB drive but need help in how to keep the system updated moving forward.

User avatar
rpdom
Posts: 11522
Joined: Sun May 06, 2012 5:17 am
Location: Essex, UK

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Nov 22, 2016 7:36 am

anderml1955 wrote:I suspected updating the system with those would break the boot process so I cloned the USB drive and ran the update. As expected the drive does not boot. I'm now running on my original USB drive but need help in how to keep the system updated moving forward.
Yes, you need to stop the system from installing the raspberrypi-bootloader and raspberrypi-kernel packages as (currently) they will overwrite the versions that you installed using rpi-update with versions that don't support USB boot.

I don't know how to do it in webmin, but on the command line I would do

Code: Select all

sudo apt-mark hold raspberrypi-bootloader raspberrypi-kernel
and that will prevent the packages being updated by apt-get. Once the new boot options make it into the mainstream kernel and bootloader you can "unhold" then and revert to the normal updates.

User avatar
DougieLawson
Posts: 29736
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Nov 22, 2016 8:16 am

If you have got the stock kernel installed by apt-get use
sudo rm /boot/.firmware_revision && sudo BRANCH=next rpi-update
to get the USB/PXE bootcode back on your system.
Microprocessor, Raspberry Pi & Arduino Hacker
Mainframe database troubleshooter
MQTT Evangelist
Twitter: @DougieLawson

Since 2012: 1B*5, 2B*2, B+, A+, Zero*2, 3B*3

Please post ALL technical questions on the forum. Do not send private messages.

anderml1955
Posts: 5
Joined: Tue Nov 22, 2016 2:55 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Nov 24, 2016 1:34 am

rpdom wrote:
anderml1955 wrote:I suspected updating the system with those would break the boot process so I cloned the USB drive and ran the update. As expected the drive does not boot. I'm now running on my original USB drive but need help in how to keep the system updated moving forward.
Yes, you need to stop the system from installing the raspberrypi-bootloader and raspberrypi-kernel packages as (currently) they will overwrite the versions that you installed using rpi-update with versions that don't support USB boot.

I don't know how to do it in webmin, but on the command line I would do

Code: Select all

sudo apt-mark hold raspberrypi-bootloader raspberrypi-kernel
and that will prevent the packages being updated by apt-get. Once the new boot options make it into the mainstream kernel and bootloader you can "unhold" then and revert to the normal updates.
Thanks very much. I'll do that now and see if webmin honors it, if not I'll turn that off and use the command line.

anderml1955
Posts: 5
Joined: Tue Nov 22, 2016 2:55 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Nov 24, 2016 1:35 am

DougieLawson wrote:If you have got the stock kernel installed by apt-get use
sudo rm /boot/.firmware_revision && sudo BRANCH=next rpi-update
to get the USB/PXE bootcode back on your system.
Great info. I'll link this to the comments on the USB boot page.

comdif
Posts: 3
Joined: Tue Jan 03, 2017 8:31 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Jan 03, 2017 11:19 pm

First I want to say thank's for this new feature.
I have tested the first raspberry some years ago and just put it in the trash after some tests:
Very slow, sd card easy corrupted, sensibility with the power supply, ... not reliable for production environments

For me the first advantage of a Linux environment is to be plugged/unplugged without any care
then to change hardware and drive also with not any care and have always an operational system.
This is the case for most traditional servers and also for many embedded system.
I have now a long experience on optware and optware-ng working on DD-wrt arm routers on an USB drive formated EXT2
and have many systems in production with hard conditions working without problems since many years.

PI3 change my mind for the first problem (slow) but not for the others as my first tests with it destroy my micro SD in 3 days
raspi powered with a good POE extractor isolated.
(Please Raspberry foundation make effort to build POE embedded for next generation because it makes sense with a small embedded hardware)

I hope the USB booting feature will make this hardware strong and reliable I'll make many hard tests
If I can have a suggestion it could be a good idea to offer a sh script to make all the job for newbies
this is what I will do :-)

thegnnu
Posts: 155
Joined: Thu Oct 18, 2012 7:07 pm
Location: Bristol

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jan 06, 2017 12:36 pm

Hi Can anybody give explain the format to use UUID in fstab
instead of the /dev/sda1 for the boot partition as using 1 usb pen 2 HD in raid the /dev/sda can change to /dev/sdc if you remove usb pen then when plugging it back in it comes out as /dev/sdc ending up in kernel panic cannot find /root

TerryR

gkreidl
Posts: 5245
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jan 06, 2017 12:55 pm

thegnnu wrote:Hi Can anybody give explain the format to use UUID in fstab
instead of the /dev/sda1 for the boot partition as using 1 usb pen 2 HD in raid the /dev/sda can change to /dev/sdc if you remove usb pen then when plugging it back in it comes out as /dev/sdc ending up in kernel panic cannot find /root

TerryR
Check the extended method in the OP.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

Return to “Advanced users”

Who is online

Users browsing this forum: No registered users and 6 guests