User avatar
GTR2Fan
Posts: 1601
Joined: Sun Feb 23, 2014 9:20 pm
Location: South East UK

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 08, 2016 9:33 pm

bodge99 wrote:Yes, OK, using a SSD is total overkill for this.. but I had a 128GB m.2 drive spare after a tablet drive upgrade. I popped it into a USB caddy and went from there. I don't need any external power as the Pi's USB supply is enough.
I don't think it's overkill at all as it must be giving you excellent small file read and write speeds that you're probably never going to see from a USB flash drive. Divide the cost between the number of machines it services and it makes perfect sense to me.

I'd love to see a guide. Go for it! :)
Pi2B Mini-PC/Media Centre: ARM=1GHz (+3), Core=500MHz, v3d=500MHz, h264=333MHz, RAM=DDR2-1200 (+6/+4/+4+schmoo). Sandisk Ultra HC-I 32GB microSD card on '50=100' OCed slot (42MB/s read) running Raspbian/KODI16, Seagate 3.5" 1.5TB HDD mass storage.

User avatar
bodge99
Posts: 20
Joined: Tue Mar 08, 2016 7:09 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 08, 2016 9:39 pm

Hi,

Thanks for the welcome.. appreciated!!

I'll write something up over the next couple of days and make a fresh thread.

It might be an idea to leave my original post in place as it is somewhat connected and might give further pointers to others.

More soon..

Bodge99

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 08, 2016 9:46 pm

bodge99 wrote: I run multiple generation Pi's from one SSD drive.. although obviously not at the same time!!
The SSD is currently partitioned into 5 and I've got 5 totally independent installations to work with.
If this drive was on a network, you could use them all at the same time on five different pi. But it sounds like you've got it plugged into just the one pi directly. You could still share from there to the other pi on a network and get them all going at once.

Swiftek
Posts: 23
Joined: Fri May 25, 2012 12:10 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 08, 2016 9:48 pm

stderr wrote:
bodge99 wrote: I run multiple generation Pi's from one SSD drive.. although obviously not at the same time!!
The SSD is currently partitioned into 5 and I've got 5 totally independent installations to work with.
If this drive was on a network, you could use them all at the same time on five different pi. But it sounds like you've got it plugged into just the one pi directly. You could still share from there to the other pi on a network and get them all going at once.
Sharing the root filesystem of a server on the network? Is that wise?
EDIT: I'm assuming it's a server, which I guess I have no reason to do.

User avatar
bodge99
Posts: 20
Joined: Tue Mar 08, 2016 7:09 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 08, 2016 10:08 pm

Hi,

Just to clarify... The SSD is in a USB caddy which is then connected to the particular Pi that I'm working with.

I "got into" Raspberry Pi's a while ago. I had dumped windows a couple of years before this and have been running Linux on all of my hardware since then.

Current projects include working to improve Linux hardware support on tablets (the Linx range, an Onda 10" and a Cube i7 Stylus). I'm also looking at coreboot on intel powered netbooks (Acer D255, D270 and the HP mini 210 4000 series) .
I've found the Pi to be perfect as a SPI flashchip programmer (with flashrom).

You'll see my tablet firmware reflashing guide using a Raspberry Pi on one of the tablet forums..

Bodge99

User avatar
GTR2Fan
Posts: 1601
Joined: Sun Feb 23, 2014 9:20 pm
Location: South East UK

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 08, 2016 10:28 pm

bodge99 wrote:Current projects include working to improve Linux hardware support on tablets (the Linx range...
Ooh! That would be of particular interest to me as I have a Linx7 running Win10 that I'd rather run Linux on if it might improve the current 4 hours per charge I'm getting. A PM with details would be much appreciated if you get the time. :)
Pi2B Mini-PC/Media Centre: ARM=1GHz (+3), Core=500MHz, v3d=500MHz, h264=333MHz, RAM=DDR2-1200 (+6/+4/+4+schmoo). Sandisk Ultra HC-I 32GB microSD card on '50=100' OCed slot (42MB/s read) running Raspbian/KODI16, Seagate 3.5" 1.5TB HDD mass storage.

User avatar
bodge99
Posts: 20
Joined: Tue Mar 08, 2016 7:09 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 08, 2016 11:10 pm

Hi,

Sorry, slightly off topic here

"Mainstream" Linux support for the Linx range is still very much "work in progress". Support for the tablet soc (bay trail-t) is fairly good with the latest kernels but most of the hardware support is still not finished.

I've fully sorted the booting (64bit OS from a 32bit UEFI), the screen (unaccelerated, for now) and digitizer work.
Wifi and bluetooth are wip (nearly there!). I've had some success at accessing the accelerometer (for auto rotation) and audio is coming along nicely (unstable at this time). I haven't yet looked at the cameras or the battery hardware.

The tenuous link to the Pi here is that I use the Pi's SPI for firmware flashing and with a home bodged (very crude) logic analyzer.

Bodge99.

greenbag
Posts: 121
Joined: Wed Aug 05, 2015 12:52 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Mar 22, 2016 11:12 am

I did it an easier way... I just wrote the image to both disks, then edited the cmdline.txt on the card to direct to /dev/sda2. The only issue I had, was when I wanted to edit the /boot/config.txt file... the taskbar would flicker and disappear on reboot. I edited the config.txt on the external drive by commenting everything out.. I figured one was screwing the other up. Other than that.. it's been running as expected.

User avatar
Nastanir
Posts: 33
Joined: Tue Dec 24, 2013 3:16 pm
Location: IIT Guwahati, India
Contact: Website

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Apr 03, 2016 2:30 pm

The method worked again on Jessie today. I have blogged a edited version of this method with a link to this posting. https://nastanir.wordpress.com/2016/04/ ... usb-stick/. Hope I am not violating any copyright.

MrKoolics
Posts: 15
Joined: Wed Feb 03, 2016 1:02 am
Location: Twin Cities, MN

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sat Apr 09, 2016 2:37 pm

I just purchased a Raspberry Pi 3 and followed this tutorial and I think it was great. I have a couple of questions though. How do I know if the file system is running off the USB HDD now? Here is the output of my df -h:
Filesystem Size Used Avail Use% Mounted on
/dev/root 917G 4.5G 866G 1% /
devtmpfs 364M 0 364M 0% /dev
tmpfs 368M 26M 343M 7% /dev/shm
tmpfs 368M 5.4M 363M 2% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 368M 0 368M 0% /sys/fs/cgroup
/dev/mmcblk0p1 60M 20M 41M 34% /boot
tmpfs 74M 0 74M 0% /run/user/1000

I see there is a /dev/root but no rootfs entry? Also, before I followed this tutorial I never noticed the green LED blinking on the Pi3. Now that green LED system to continue blinking. Does anyone have any ideas?

I will say I'm not a computer programmer by any means. I typically use Linux because I'm looking for cheap, viable alternatives to the more expensive competitors. I'm also using the Raspbian Jesse downloaded from the Raspberrypi.org website. Here is the cmdline.txt:

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait rootdelay=5

I just want to make sure that I followed the tutorial appropriately and that I won't still corrupt the SD card. I did a similiar exercise on a Pi2 but don't recall the GREEN LED continuing to blink after boot up.

Thanks for any help that anyone can offer. Have a great day.
Last edited by MrKoolics on Sun Apr 10, 2016 6:45 pm, edited 1 time in total.
Raspberry Pi 2 Model B w/ OSMC & Raspberry Pi 3 Model B w/ Raspbian Jesse

levelcrow
Posts: 47
Joined: Tue Aug 12, 2014 5:04 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Apr 10, 2016 5:33 pm

I've got a script I wrote to move root onto a USB drive for a F2FS filesystem. I don't think I saw a script so far in this thread for F2FS, and the instructions seem to be specialized for EXT4. The script handles drives by PARTUUID; it generates a cmdline.txt file in /boot and also modifies fstab on the target drive.

Instructions
NOTE: Please back up all data before using this script. In particular, make sure you back up any data on your target USB drive that you intend to keep. Running gparted with partition changes or this script on the target USB drive can wipe out any and all data on the USB drive.

1. Install f2fs-tools, rsync, and gparted. In terminal:

Code: Select all

sudo apt-get update && sudo apt-get install f2fs-tools rsync gparted
2. Start gparted and create an adequately sized (larger than your current root folder) f2fs partition on your target USB drive. This script is specific for f2fs target partitions and won't boot if the target partition isn't f2fs - this is because of the fstab file that is generated. If you want to modify the script to work with different filesystems, you just need to change the f2fs designation in the script. Take note of what you name the partition. Unplug and replug the USB drive such that the new partition is mounted.

3. Save the following code somewhere as backup.py

Code: Select all

#! /usr/bin/env python

import os, subprocess,sys,time

white = '\033[1;37m'
default = '\033[0;37m'
red = '\033[0;31m'
		
drives = []
if(len(sys.argv)==1):
	entry = raw_input(white+"What is the name of the partition you would like to back up on?\n\t"+default)
else:
	entry = sys.argv[1]

drives.append(entry)
		
filedir = os.path.dirname(os.path.abspath(__file__)) + '/'
d = subprocess.check_output(['df','-h'],shell=True)

d = d.replace("\n"," ")
driveslist = d.split(" ")
while(driveslist.count("")!=0):
	driveslist.remove("")

devname = []
partname = []

copy = False
for i in range(len(driveslist)):
	if(driveslist[i].count("/dev/")!=0 and driveslist[i-1].count("%")==0):
		devname.append(driveslist[i])
		copy = True
	if(driveslist[i].count("%")!= 0 and driveslist[i-1].isdigit() and copy):
		partname.append(driveslist[i+1])
		copy = False

i = 0
while(i != len(partname)):
	if('/media/pi' not in partname[i]):
		partname.remove(partname[i])
		devname.remove(devname[i])
	else:
		partname[i]=partname[i][len('/media/pi/'):]
		i+=1

partition = ""
dev = ""
for i in partname:
	if(drives.count(i)!=0):
		partition = i
		dev = devname[partname.index(i)]
		print(i)
		print(dev)
		break
else:
	sys.exit(red+"No valid drives found"+default)
	
uuid = ""
puuid = ""
blkid = subprocess.check_output(['sudo','blkid',dev])
blkid = blkid[blkid.index('UUID="')+6:]
uuid = blkid[:blkid.index('"')]
blkid = blkid[blkid.index('UUID="')+6:]
puuid = blkid[:blkid.index('"')]

subprocess.call(['sudo','umount',dev])
subprocess.call(['sudo','mount',dev,'/mnt'])
subprocess.call(['sudo','rsync','-axv','--delete','/','/mnt'])

f = open('/home/pi/'+partition+'-cmdline.txt','w')
f.write('dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=PARTUUID='+puuid+' rootfstype=f2fs elevator=deadline rootwait rootdelay=5')
f.close()
f = open('/home/pi/fstab','w')
f.write('proc            /proc           proc    defaults          0       0\n/dev/mmcblk0p1  /boot           vfat    defaults          0       2\n#/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1\n# a swapfile is not a swap partition, no line here\n#   use  dphys-swapfile swap[on|off]  for that\n/dev/disk/by-uuid/'+uuid+'    /   f2fs    defaults,noatime  0       0')
f.close()

subprocess.call(['sudo','mv','/home/pi/fstab','/mnt/etc/fstab'])
subprocess.call(['sudo','mv','/home/pi/'+partition+'-cmdline.txt','/boot/'+partition+'-cmdline.txt'])

subprocess.call(['sudo','umount','/mnt'])

4. Run the script by navigating to the directory containing backup.py in terminal and then entering "python backup.py"

5. Enter the name of the partition. For example, if the partition is mounted as /media/pi/partition_name , then partition_name is input.

6. After the script completes, a new file should be present in /boot containing the cmdline.txt specific for the new backup. The name of the cmdline.txt variant is the partition name preceding cmdline.txt. Using the previous example, the filename should be partition_name-cmdline.txt. Copying this new cmdline.txt variant over the cmdline.txt will change the root directory to be shifted over to the USB drive on next boot. It's probably a good idea to back up the old cmdline.txt for convenience.

Code: Select all

sudo cp /boot/ cmdline.txt backup-cmdline.txt
sudo cp (partition name)-cmdline.txt cmdline.txt
***

That should be it. Tell me if it isn't working. I like the script a lot and use it often. The script allows for arguments and so you can conveniently set up a series of backups like this:
python backup.py partition1 && python backup.py partition2 && python backup.py partition3

johnmx
Posts: 1
Joined: Wed Sep 11, 2013 2:33 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Apr 20, 2016 1:14 am

I followed the above tutorial made by levelcrow. It worked but there is something strange in my system. The 4k random read is slower than the write for the HDD, and both speeds are slower compared to when the root is in the internal microSD card!
I used an RPi3 in 3 different scenarios:

Code: Select all

(used cmd: iozone3_434/src/current/iozone -e -I -a -s 100M -r 4k -i 0 -i 1 -i 2)

 1 - Root in the internal microSD (no brand cheap 32G Class 10 card):
     4K rand read: 5.42 MB/sec
     4K rand write: 1.14 MB/sec

 2 - Root in an external HDD “WD My passport 500GB” (ext4):
     4K rand read: 0.50 MB/sec (very strange result)
     4K rand write: 0.94 MB/sec

 3 - Root in a Samsung 128GB MUF-128CB USB 3.0 Duo Drive (f2fs):
     4K rand read: 3.78 MB/sec
     4K rand write: 0.81 MB/sec
In scenario 1 (root in microSD) I was able to get good results accessing the HDD and the USB PEN:

Code: Select all

HDD “WD My passport 500GB” (ext4):
   4K rand read: 67.54 MB/sec (too high, maybe the command is not correct for this case??)
   4K rand write: 15.07 MB/sec

Samsung 128GB MUF-128CB USB 3.0 Duo Drive (f2fs):
   4K rand read: 10.33 MB/sec
   4K rand write: 6.51 MB/sec
Does anyone know why the speeds are so slow?

Best regards

gkreidl
Posts: 5302
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Apr 20, 2016 5:45 am

Obviously the test reports crap. From real world applications (compiling large packages, for example) I know, that USB HDD is faster than the fastest SD cards with overclocked SD interface.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

greenbag
Posts: 121
Joined: Wed Aug 05, 2015 12:52 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Apr 27, 2016 7:00 pm

gkreidl wrote:Obviously the test reports crap. From real world applications (compiling large packages, for example) I know, that USB HDD is faster than the fastest SD cards with overclocked SD interface.
Yep.. usb hard drive is about 2x faster than the sd card. USB ssd's even faster. :)

greenbag
Posts: 121
Joined: Wed Aug 05, 2015 12:52 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Apr 27, 2016 7:05 pm

Seriously... the easiest way to do this, is to write the image to both the microsd, and to the external drive. Edit sd card's /boot/cmdline.txt to point to root=/dev/sda2. Reboot. That's all that's needed. You might want to resize the root partition of the external drive using gparted first though.

This is my cmdline.txt from Ubuntu 14.04.

Code: Select all

dwc_otg.lpm_enable=0 console=tty1 root=/dev/sda2 rootwait
It's seriously that easy.

User avatar
bodge99
Posts: 20
Joined: Tue Mar 08, 2016 7:09 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Apr 27, 2016 8:10 pm

Yes, but if you add or move a device/partition etc. then the system might id your partition differently..
That's why UUIDs/PARTUUIDs are used.

greenbag
Posts: 121
Joined: Wed Aug 05, 2015 12:52 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Apr 27, 2016 10:27 pm

bodge99 wrote:Yes, but if you add or move a device/partition etc. then the system might id your partition differently..
That's why UUIDs/PARTUUIDs are used.
Then use UUID's. The process is still the same... no need to copy the entire fs from1 disk to the other.. just redirect cmdline.txt to the root=UUID whetever it is.

User avatar
bodge99
Posts: 20
Joined: Tue Mar 08, 2016 7:09 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Apr 27, 2016 11:26 pm

Yep, you are quite right of course..

I personally prefer to create the partitions and filesystems manually. I then populate the filesystems after performing hardware checks. This is just a method that I've evolved over time..

"Horses for courses.." ;)

greenbag
Posts: 121
Joined: Wed Aug 05, 2015 12:52 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Apr 28, 2016 2:03 pm

bodge99 wrote:Yep, you are quite right of course..

I personally prefer to create the partitions and filesystems manually. I then populate the filesystems after performing hardware checks. This is just a method that I've evolved over time..

"Horses for courses.." ;)
I'm lazy... lol. :P

chris.gill
Posts: 1
Joined: Thu May 12, 2016 8:56 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu May 12, 2016 9:08 pm

Thanks for this HOWTO - really helpful.

One small suggestion: On my RPi2 I was unable to specify the root partition using the PARTUUID format in the way you suggested. Like levelcrow I found that I needed to use a string obtained from the blkid command instead.

paulv
Posts: 525
Joined: Tue Jan 15, 2013 12:10 pm
Location: Netherlands

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri May 13, 2016 6:17 am

Thank you chris, I have added this to the script.

mgherghina
Posts: 4
Joined: Sat Jun 25, 2016 8:41 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sat Jun 25, 2016 8:44 pm

May i ask what the R/W speeds are on the f2fs filesystem ? Im interested in doing it , especially since i use a samsung micro usb adapter to keep my pi as low profile as possible. Does it feel snappier? i remember when i had my nexus 4 and i formatted the system partition to f2fs and it felt like a different phone. Also had a more efficient way of handling the battery.

PicoPi
Posts: 71
Joined: Sat Jun 27, 2015 10:37 pm
Location: Ottawa, Canada

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Jun 26, 2016 1:07 am

Fantastic write-up. Got it working the first time.
Excellent job.

mgherghina
Posts: 4
Joined: Sat Jun 25, 2016 8:41 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Jun 26, 2016 3:17 am

I don't understand how to use the script, it keeps on asking me for a name but I've tried everything... I'm using Dietpi as my distro .. usb comes up as sda1.. even after formatting it with disk and labeling it as "disk" it still won't work

SonOfAMotherlessGoat
Posts: 690
Joined: Tue Jun 16, 2015 6:01 am

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sun Jun 26, 2016 3:32 am

mgherghina wrote:I'm using Dietpi as my distro
That distro has it's own utilities to move to USB. Check the DietPi website.
Account Inactive

Return to “Advanced users”

Who is online

Users browsing this forum: No registered users and 19 guests