PjV
Posts: 14
Joined: Tue Dec 13, 2016 5:39 pm

command line version of piclone?

Wed Apr 12, 2017 10:31 am

Hi,

My query in short: Is there a command line version of piclone?
(see: https://www.raspberrypi.org/blog/anothe ... -raspbian/ scroll down to SD card copier)
I have a 'headless' PI running Jessie Lite, so I can't run piclone from the UI. Via SSH I can access the device.

(Edit: it's piclone, not picopy)
Last edited by PjV on Wed Apr 12, 2017 3:30 pm, edited 1 time in total.

scotty101
Posts: 4203
Joined: Fri Jun 08, 2012 6:03 pm

Re: command line version of picopy?

Wed Apr 12, 2017 10:38 am

Yes. it's called dd and is a standard *nix tool and how we used to copy .img files to SD cards back in the old days.
Electronic and Computer Engineer
Pi Interests: Home Automation, IOT, Python and Tkinter

mikerr
Posts: 2826
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: command line version of picopy?

Wed Apr 12, 2017 10:49 am

picopy ("piclone") isn't the same as dd

dd is a direct byte for byte copy - so resulting .img would be huge, and also can't copy to smaller sd cards

piclone first sets up partitons, then does a file based copy - so can move a 32GB SD card install to 8 GB etc..

In the github for piclone, there is the original command line script here:

https://github.com/raspberrypi-ui/piclo ... src/backup
Android app - Raspi Card Imager - download and image SD cards - No PC required !

PjV
Posts: 14
Joined: Tue Dec 13, 2016 5:39 pm

Re: command line version of piclone?

Wed Apr 12, 2017 3:32 pm

Thanks Mike,

I corrected the name of the tool.
The link you provided looks to be what I needed :)

PlanB
Posts: 102
Joined: Tue Oct 28, 2014 6:44 am

Re: command line version of piclone?

Mon Jun 05, 2017 12:28 pm

I pasted the above code into a file I named sdclone & tried to run it. I got.....
root@raspberrypi:/home/pi# bash sdclone
umount: 9: mountpoint not found
umount: 8: mountpoint not found
umount: 7: mountpoint not found
umount: 6: mountpoint not found
umount: 5: mountpoint not found
umount: 4: mountpoint not found
umount: 3: mountpoint not found
umount: 2: mountpoint not found
umount: 1: mountpoint not found
dd: failed to open ‘’: No such file or directory
Error: Could not stat device mklabel - No such file or directory.
Retry/Cancel?
Clearly my simple minded approach has missed some detail. Is there a simple guide for how to get piclone working on Jessie lite where there is no startx or pixel?

SkyRise
Posts: 179
Joined: Tue Jan 24, 2012 1:20 pm

Re: command line version of piclone?

Mon Jun 05, 2017 12:40 pm

The script requires the target device on the command line

Code: Select all

sdclone /dev/sdx
Make sure to get /dev/sdx right, or (as usual) you could wipe out your pi...
(oh, and will probably need to be run as root too)

PlanB
Posts: 102
Joined: Tue Oct 28, 2014 6:44 am

Re: command line version of piclone?

Mon Jun 05, 2017 1:16 pm

Thnx for the tip, any idea how to fix the 'operation not permitted' errors?

root@raspberrypi:/home/pi# bash sdclone /dev/sda
umount: /dev/sda9: mountpoint not found
umount: /dev/sda8: mountpoint not found
umount: /dev/sda7: mountpoint not found
umount: /dev/sda6: mountpoint not found
umount: /dev/sda5: mountpoint not found
umount: /dev/sda4: mountpoint not found
umount: /dev/sda3: mountpoint not found
umount: /dev/sda2: not mounted
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.0219021 s, 23.4 kB/s
Information: You may need to update /etc/fstab.

Information: You may need to update /etc/fstab.

mkfs.fat 3.0.27 (2014-11-12)
mkfs.fat: /dev/sda1 contains a mounted filesystem.
‘/tmp/tmp.yDOGKpbbDU/./COPYING.linux’ -> ‘/tmp/tmp.p8jDE1pUJ8/././COPYING.linux’
cp: failed to preserve ownership for ‘/tmp/tmp.p8jDE1pUJ8/././COPYING.linux’: Operation not permitted
‘/tmp/tmp.yDOGKpbbDU/./LICENCE.broadcom’ -> ‘/tmp/tmp.p8jDE1pUJ8/././LICENCE.broadcom’
cp: failed to preserve ownership for ‘/tmp/tmp.p8jDE1pUJ8/././LICENCE.broadcom’: Operation not permitted
‘/tmp/tmp.yDOGKpbbDU/./bcm2708-rpi-b-plus.dtb’ -> ‘/tmp/tmp.p8jDE1pUJ8/././bcm2708-rpi-b-plus.dtb’
cp: failed to preserve ownership for ‘/tmp/tmp.p8jDE1pUJ8/././bcm2708-rpi-b-plus.dtb’: Operation not permitted
‘/tmp/tmp.yDOGKpbbDU/./bcm2708-rpi-b.dtb’ -> ‘/tmp/tmp.p8jDE1pUJ8/././bcm2708-rpi-b.dtb’
cp: failed to preserve ownership for ‘/tmp/tmp.p8jDE1pUJ8/././bcm2708-rpi-b.dtb’: Operation not permitted
‘/tmp/tmp.yDOGKpbbDU/./bcm2709-rpi-2-b.dtb’ -> ‘/tmp/tmp.p8jDE1pUJ8/././bcm2709-rpi-2-b.dtb’

PlanB
Posts: 102
Joined: Tue Oct 28, 2014 6:44 am

Re: command line version of piclone?

Mon Jun 05, 2017 1:17 pm

I get a similar gripe when I try to use Bill Wilson's rpi-clone

User avatar
rpdom
Posts: 18148
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: command line version of piclone?

Mon Jun 05, 2017 1:47 pm

PlanB wrote:Thnx for the tip, any idea how to fix the 'operation not permitted' errors?
Just ignore them. They're harmless. It's just the copy trying to set the ownership and permissions on the files, but the VFAT file system in /boot doesn't support them.

User avatar
peterscargill
Posts: 16
Joined: Fri Apr 17, 2015 10:40 pm
Location: Northumberland
Contact: Website

Re: command line version of piclone?

Thu Jul 12, 2018 3:02 pm

Interesting responses in here including the smartalec DD response. PICLONE is definitely NOT DD.

I got mountpoint not found for sda9 through 3, but that is to be expected. sda2 and 1 not found. re-reading partition table failed - resource busy, HOWEVER, it DOES seem to be working, now half way through partition 2 which begs the question why the warnings and apparent error on rereading the table?

Desktop CD Copy has always worked for me and so I've no reason to think this command line version won;t... ignoring those mountpoint warnings, is that re-read failed message is maybe important? Dialog says copy complete. Rebooted onto new SD. Works.. but that message is bugging me.

Did this on a PI2 with full Raspbian.
Regards

Peter Scargill (tech.scargill.net)

dckornbluh
Posts: 27
Joined: Fri Mar 03, 2017 9:31 pm
Location: Seattle, WA

Re: command line version of piclone?

Sat Dec 15, 2018 11:14 pm

I used sdclone successfully (see github link above) and it got me out of a pickle. I have an RPi model 3A+ which has only one USB port. This enabled me to SSH in and successfully clone the current SD card.

Tips for newbies: use command lsblk which yields
  • NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
    sda 8:0 1 7.4G 0 disk
    ├─sda1 8:1 1 41.8M 0 part
    └─sda2 8:2 1 7.4G 0 part
    mmcblk0 179:0 0 7.4G 0 disk
    ├─mmcblk0p1 179:1 0 1.8G 0 part /tmp/tmp.egPTiZW7gf
    ├─mmcblk0p5 179:5 0 32M 0 part
    ├─mmcblk0p6 179:6 0 69M 0 part /boot
    └─mmcblk0p7 179:7 0 5.6G 0 part /
Then I used this command:

Code: Select all

sudo ./sdclone /dev/sda /dev/macblk0

User avatar
HawaiianPi
Posts: 6216
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: command line version of piclone?

Sun Dec 16, 2018 2:22 am

dckornbluh wrote:
Sat Dec 15, 2018 11:14 pm
Then I used this command:

Code: Select all

sudo ./sdclone /dev/sda /dev/macblk0
Shouldn't that be mmcblk0 (not macblk0)?

Code: Select all

sudo ./sdclone /dev/sda /dev/mmcblk0
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

huongminh
Posts: 1
Joined: Thu May 16, 2019 11:06 am

Re: command line version of piclone? cloned card does not boot

Thu May 16, 2019 11:10 am

I follow steps below and successfully clone my sd card. However, the cloned card does not boot. Please help!


dckornbluh wrote:
Sat Dec 15, 2018 11:14 pm
I used sdclone successfully (see github link above) and it got me out of a pickle. I have an RPi model 3A+ which has only one USB port. This enabled me to SSH in and successfully clone the current SD card.

Tips for newbies: use command lsblk which yields
  • NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
    sda 8:0 1 7.4G 0 disk
    ├─sda1 8:1 1 41.8M 0 part
    └─sda2 8:2 1 7.4G 0 part
    mmcblk0 179:0 0 7.4G 0 disk
    ├─mmcblk0p1 179:1 0 1.8G 0 part /tmp/tmp.egPTiZW7gf
    ├─mmcblk0p5 179:5 0 32M 0 part
    ├─mmcblk0p6 179:6 0 69M 0 part /boot
    └─mmcblk0p7 179:7 0 5.6G 0 part /
Then I used this command:

Code: Select all

sudo ./sdclone /dev/sda /dev/macblk0

Bosse_B
Posts: 1086
Joined: Thu Jan 30, 2014 9:53 am

Re: command line version of piclone?

Wed Feb 26, 2020 11:56 pm

I know this thread has been silent for a while, but I hope I can get advice on some small details....

I am usually working with non-GUI RPi units and it is a PITB making backup copies of the SD-card on a headless RPi only accessing it via SSH.

So I used the link above to retrieve the backup script from github, here renamed to sdclone.
When I looked inside the script I could not see it using more than one argument and that is the target drive name, so the correct usage should be:

Code: Select all

sudo ./sdclone /dev/sda
The code sets the src and dst variables internally like this:

Code: Select all

src=$(mktemp -d)
dst=$(mktemp -d)
What does this in fact do?
They are used later in the code in mount, cp and umount operations:

Code: Select all

sudo mount $1$pnum $dst
sudo mount /dev/mmcblk0p$pnum $src
sudo cp -axv $src/. $dst/.
sudo umount $dst
sudo umount $src
I am curious how the setting works to wind up with the correct values....
Bo Berglund
Sweden

GlowInTheDark
Posts: 1370
Joined: Sat Nov 09, 2019 12:14 pm

Re: command line version of piclone?

Thu Feb 27, 2020 12:51 am

The code sets the src and dst variables internally like this:

Code: Select all

src=$(mktemp -d)
dst=$(mktemp -d)
What does this in fact do?
ObSnarkiness: "man mktemp"

It creates a guaranteed unique directory under /tmp and returns the name of the new directory.

BTW, earlier up in your post, you say:
sudo ./sdclone /dev/sda
I'm pretty sure the "sudo" is unnecessary. I'm basing this on the fact that most of the script's commands that need it seem to be preceded with pleasesudo on their own, so it shouldn't be necessary to sudo the script itself.

Note that seeing pleasesudo preceding a shell script kinda kicked off my "setuid shell scripts are a bad idea" alarm buzzer (about which I could write much, but you can do the research yourself if you care to), but here it seems to be OK.
GitD's list of things that are not ready for prime time:
1) IPv6
2) 64 bit OSes
3) USB 3
4) Bluetooth

Loves Linux; loves to dance.

Bosse_B
Posts: 1086
Joined: Thu Jan 30, 2014 9:53 am

Re: command line version of piclone?

Thu Feb 27, 2020 7:52 am

GlowInTheDark wrote:
Thu Feb 27, 2020 12:51 am
BTW, earlier up in your post, you say:
sudo ./sdclone /dev/sda
I'm pretty sure the "sudo" is unnecessary. I'm basing this on the fact that most of the script's commands that need it seem to be preceded with sudo on their own, so it shouldn't be necessary to sudo the script itself.
Correct, I just quoted the original suggestion for the command execution, but if I look closer in the script it precedes all such commands with sudo that need elevation.

Now that I can understand it I will try it out on my newly configured RPi4.
Bo Berglund
Sweden

Bosse_B
Posts: 1086
Joined: Thu Jan 30, 2014 9:53 am

Re: command line version of piclone?

Thu Feb 27, 2020 9:59 am

I just came across this command line rpi-clone on github.
Has someone here tested it and can give feedback?
It seems like it has many features like the ability to be used as a backup function with incremental updates of a base copy where only changed files are transferred.

Maybe this would be a better solution than the script discussed above?
OTOH:
The rpi-clone script is huge and does a lot of different things, maybe "keep it simple, stupid" is a valid comment here?
Bo Berglund
Sweden

Bosse_B
Posts: 1086
Joined: Thu Jan 30, 2014 9:53 am

Re: command line version of piclone?

Thu Feb 27, 2020 5:18 pm

I went ahead and inserted an 8GB SD-card into a USB carrier and inserted it into my RPi4 where I had already put the sdclone script file into my bin dir.
I ran it using the command:

Code: Select all

sdclone /dev/sda
After about 20 min it was done and displayed a message concerning some fix that needed to be done to fstab.
Since I did not know what that meant I just shut down my RPi4 and switched SDcard so the clone is now in the RPi4.
But on startup nothing much happens, it does not come on line so I can access it via PuTTY (I have a boot script that posts the IP address of the RPi following boot and this does not happen.

What can I look for now?

If I switch the old SDcard back to the RPi4 it starts up OK and if I insert the USB caddy with the cloned card I get the following:

Code: Select all

lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 14cd:1212 Super Top microSD card reader (SY-T18) <== My SDcard reader
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

sudo lsblk -o UUID,NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL,MODEL
UUID                                 NAME        FSTYPE  SIZE MOUNTPOINT LABEL  MODEL
                                     sda                 7.4G                   Storage_Device
8139-B4CA                            ├─sda1      vfat    256M
e31a2667-663c-4715-a994-bb94af77a1b7 └─sda2      ext4    7.2G
                                     mmcblk0            28.9G
69D5-9B27                            ├─mmcblk0p1 vfat    256M /boot      boot
24eaa08b-10f2-49e0-8283-359f7eb1a0b6 └─mmcblk0p2 ext4   28.7G /          rootfs
So at least the partitions on the new card looks OK. Note that I used an 8GB card when cloning from the 32 GB original since the actual used space on the card was only some 2.7 GB (it is the lite version of Raspbian Buster).
Regarding fstab, here is what it contains on the running RPi4:

Code: Select all

cat /etc/fstab
proc            /proc           proc    defaults          0       0
PARTUUID=d9b3f436-01  /boot           vfat    defaults          0       2
PARTUUID=d9b3f436-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
I mounted the SDcard in the caddy and this is what fstab on it looks like:

Code: Select all

cat /mnt/sdroot/etc/fstab
proc            /proc           proc    defaults          0       0
PARTUUID=d9b3f436-01  /boot           vfat    defaults          0       2
PARTUUID=d9b3f436-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
How can I use this information to adjust fstab, if that is what is needed to make the RPi4 run with the cloned card?

Any advice welcome, it seems like I am almost there...
Bo Berglund
Sweden

Bosse_B
Posts: 1086
Joined: Thu Jan 30, 2014 9:53 am

Re: command line version of piclone?

Thu Feb 27, 2020 10:01 pm

Bosse_B wrote:
Thu Feb 27, 2020 9:59 am
I just came across this command line rpi-clone on github.
Has someone here tested it and can give feedback?
It seems like it has many features like the ability to be used as a backup function with incremental updates of a base copy where only changed files are transferred.

Maybe this would be a better solution than the script discussed above?
OTOH:
The rpi-clone script is huge and does a lot of different things, maybe "keep it simple, stupid" is a valid comment here?
SOLVED!
I can report that unlike the simpler script discussed above rpi-clone actually does its job and finishes (in my case in 24 min) with a bootable target SD-card!
This is what I will use going forward to maintain a backup.
Bo Berglund
Sweden

corbathegeek
Posts: 1
Joined: Thu Mar 19, 2020 9:45 pm

Re: command line version of picopy?

Thu Mar 19, 2020 9:48 pm

mikerr wrote:
Wed Apr 12, 2017 10:49 am
picopy ("piclone") isn't the same as dd

dd is a direct byte for byte copy - so resulting .img would be huge, and also can't copy to smaller sd cards

piclone first sets up partitons, then does a file based copy - so can move a 32GB SD card install to 8 GB etc..

In the github for piclone, there is the original command line script here:

https://github.com/raspberrypi-ui/piclo ... src/backup
So... I would run the script like this?

Code: Select all

./piclone.sh > mygolden.img

Bosse_B
Posts: 1086
Joined: Thu Jan 30, 2014 9:53 am

Re: command line version of piclone?

Fri Mar 20, 2020 9:23 am

In my case I keep the backed up data on a (smaller) SDcard, which is a clone of the active SDcard.
It took 24 min to create it using rpi-clone and I have verified that it is OK by running it in the RPi4.

Then I have reverted to the original SDcard and continued working on the RPi4.
Now I can back it up easily by reinserting the cloned disk in a USB carrier and then run

Code: Select all

sudo rpi-clone sda
A few questions are shown to define what I want to do and then it rsync's my active SD onto the copy SD.
This will update the image to be again equivalent to the active disk and this time it takes just a minute or two.
So I can use this as my backup.
Bo Berglund
Sweden

Return to “Beginners”