Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 7:54 pm

So I have an RPi at the office, which is acting up. It does no longer allow SSH access via the network so I cannot reach it. I thought that it was due to a hung system, because it did respond to ping but would not allow SSH access and a service that should run on the RPi did also not allow access.
The office is a VERY long way away and I only have network access to it by way of OpenVPN.

I thought that a power cycle would solve the problem whatever it is and today I managed to get a person on site to power cycle the unit. But it did not help... :evil:

So then I managed to get the person to power off and extract the SDcard from the system, then use an USB SD reader and plug it into another RPi which I can access via SSH.

So now I can mount the two partitions on the SDCard and read the files etc via that RPi.

What I want to do now is to create an image of the SDcard located in the USB reader on the RPi.

The problem is that all of the tools I have found are utilities that create the image of the running system.
Since I cannot get into the system when it runs this is not an option...

QUESTION:
Is there a tool somewhere I can use to create an image file from an SDCard located in an USB card reader attached to the RPi?
The target location of the image is an NFS mounted share on an Ubuntu server, from which I would then download it so I can create a disk clone and start it up on an RPi over here.
So exactly like what one would do if one is cloning the local running system....

Edit:
The tool I use to create an image file of the running system was suggested in this thread here.
And the suggestion pointed to this other thread.
But the problem is that the solution clones the running system and not an SD-card attached via USB...
Bo Berglund
Sweden

RonR
Posts: 1679
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 8:45 pm

Bosse_B wrote:
Fri Oct 02, 2020 7:54 pm
What I want to do now is to create an image of the SDcard located in the USB reader on the RPi.

The problem is that all of the tools I have found are utilities that create the image of the running system.
Since I cannot get into the system when it runs this is not an option...

QUESTION:
Is there a tool somewhere I can use to create an image file from an SDCard located in an USB card reader attached to the RPi?
The target location of the image is an NFS mounted share on an Ubuntu server, from which I would then download it so I can create a disk clone and start it up on an RPi over here.

Simply use dd:

Code: Select all

dd bs=4M if=/dev/sdX of=/pathto/file status=progress
sync

wh7qq
Posts: 1480
Joined: Thu Oct 09, 2014 2:50 am

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 8:51 pm

Maybe I misunderstood you but rather than creating a .img file that might contain the problems you are trying to fix, just create a new uSD with the latest Raspberry OS, boot it up, configure it and copy the data over from the old disk...just your data files.

LTolledo
Posts: 4092
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 9:15 pm

insert source microSD
insert target microSD
run SD card copier
copy from device [select source microSD]
copy to device [select target microSD]
click [Start]
once finished eject source and target microSD

pictorial version much better?
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 10:50 pm

I probably did not describe it well enough:
- I am located in Sweden
- The RPi that is acting up is located in Texas
- There is another RPi in the Texas office that I use for development and it works OK
- Due to Covid-19 the Texas office is abandoned, almost all are working from home
- The RPi with problems is a backup OpenVPN server for the network
- Its MAC is registered with DHCP and the IP address it gets is used on the router port forward

I need to get it operational and the only way I could do that was calling an employee to drive to the office and restart the RPi. Unfortunately this did not work, it gets into the same situation after the reboot.

So I asked him to put the SDcard from the problem RPi unit into a USB card reader and plug that into the development RPi. I could then verify that I can mount it and read files on both partitions. He was then let go and drove home.

My plan was to image the SDcard in the reader the same way the imgclone program creates an image file on my NFS server at the office. This image I could then compress and download from the office to here and create a clone of the bad RPi from.
This could then be started with a monitor and keyboard attached so I can get into it and figure out a solution to the problem. Once done I would use imgclone to make a disk image and upload it to the Texas office.
Then I would call a developer there to make the SDcard from this image and start the RPi with that inserted.

So your suggestions are not really compatible with this scenario....
I still need the tool to make an img file from the old SDcard with the img size only as big as to fit the data (so the image file would not be gigantic) which is what imgclone does, but only from the running SDcard...
Bo Berglund
Sweden

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 10:55 pm

RonR wrote:
Fri Oct 02, 2020 8:45 pm
Bosse_B wrote:
Fri Oct 02, 2020 7:54 pm
What I want to do now is to create an image of the SDcard located in the USB reader on the RPi.

The problem is that all of the tools I have found are utilities that create the image of the running system.
Since I cannot get into the system when it runs this is not an option...

QUESTION:
Is there a tool somewhere I can use to create an image file from an SDCard located in an USB card reader attached to the RPi?
The target location of the image is an NFS mounted share on an Ubuntu server, from which I would then download it so I can create a disk clone and start it up on an RPi over here.

Simply use dd:

Code: Select all

dd bs=4M if=/dev/sdX of=/pathto/file status=progress
sync
I have never used dd...
Does this make a copy of some kind on an NFS network server from a mounted disk (sda)
What format will that output file be? img?

This is how the disks look like:

Code: Select all

 $ sudo lsblk -o UUID,NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL,MODEL
UUID                                 NAME        FSTYPE  SIZE MOUNTPOINT LABEL  MODEL
                                     sda                29.8G                   SDDR-339
E5B7-FEA1                            ├─sda1      vfat   41.8M            boot
b4ea8e46-fe87-4ddd-9e94-506c37005ac5 └─sda2      ext4   29.8G
                                     mmcblk0            29.8G
70A2-8001                            ├─mmcblk0p1 vfat    256M /boot      boot
a1fafd2b-1ef0-4fe8-8ac1-ad33bbb48642 └─mmcblk0p2 ext4   29.6G /          rootfs

$ df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root       30448040 6849384  22265788  24% /
devtmpfs          441236       0    441236   0% /dev
tmpfs             474516       0    474516   0% /dev/shm
tmpfs             474516    6424    468092   2% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs             474516       0    474516   0% /sys/fs/cgroup
/dev/mmcblk0p1    258095   54461    203635  22% /boot
tmpfs              94900       0     94900   0% /run/user/109
tmpfs              94900       0     94900   0% /run/user/1000
/dev/sda1          42137   22855     19282  55% /mnt/usbboot
/dev/sda2       30681436 4101428  25270796  14% /mnt/usbroot
So sda is what I need to image (both partitions into the same image file).
Note that there is only a little more than 4Gb used on sda so the img file should be approx this size.
Bo Berglund
Sweden

RonR
Posts: 1679
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 11:06 pm

Bosse_B wrote:
Fri Oct 02, 2020 10:55 pm
I have never used dd...
Does this make a copy of some kind on an NFS network server from a mounted disk (sda)
What format will that output file be? img?
...
So sda is what I need to image (both partitions into the same image file).

dd is a standard Linux utility. It will simply copy every sector of the SD card to a file unchanged. /dev/sda is correct and encompasses the entire device. Assuming the SD card contains normal RaspiOS, it will be a normal image file suitable for putting on another SD card with Etcher. The SD card should not be mounted when running dd.

The image file will be the same size as the device. If you need to shrink the image file to put it on a smaller SD card, image-shrink will do that without any loss of data.

I've attached a couple of scripts that I use to read and write image files using dd (followed by a compare). The usage syntax is:

Code: Select all

sudo read-image device file
sudo write-image file device
Attachments
rw-image.zip
(1.16 KiB) Downloaded 5 times
Last edited by RonR on Fri Oct 02, 2020 11:27 pm, edited 1 time in total.

emma1997
Posts: 1085
Joined: Sun Nov 08, 2015 7:00 pm
Location: New England (not that old one)

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 11:19 pm

I use dd a lot to restore images from RonR's great Image-Backup tool. But be "vewy cayfoo" it's dangerous. Specially if there are other drives in the system that are prone to tragic overwrite. And even if only 2 SD make sure you know which is which. I always run lsblk first to be sure.

HvdW
Posts: 163
Joined: Tue Jun 17, 2014 12:41 pm

Re: Imaging an RPi SDcard on another RPi, how?

Fri Oct 02, 2020 11:41 pm

Read the raspberry pi documentation about backup.
It'll explain a lot.
You can also create a new image at home and write that to the usb in Texas.
When it's a RPI 3b+ it can be plugged in to the usb port, leave the SD card out and it will run.
Who knows knows
Who doesn't doesn't

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Sat Oct 03, 2020 8:14 am

HvdW wrote:
Fri Oct 02, 2020 11:41 pm
You can also create a new image at home and write that to the usb in Texas.
When it's a RPI 3b+ it can be plugged in to the usb port, leave the SD card out and it will run.
My aim is to get the Texas RPi moved via the Internet to Sweden so I can run it here and examine with a real console and keyboard what is going on inside it. Needed since it won't allow SSH access for some reason. It has worked for a number of years, but stopped recently.
Then either get the important stuff (like certificates and keys for OpenVPN) off of it and put it on to a new SDcard with PiOS rather than the old Raspbian it runs now or else (if it is a simple error) fix it in place and then upload to Texas again.
I probably should start by examining the logs...

I don't know what Pi version this box is, how can I find out?
Since it is now switched off and robbed of its SDcard which is in the USB reader on another RPi....
Is there some remnant of that info in some log file or the like?
Bo Berglund
Sweden

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Sat Oct 03, 2020 8:51 am

Bosse_B wrote:
Sat Oct 03, 2020 8:14 am
I probably should start by examining the logs...
I think I have found some reason that the RPi is not responding as it should when I look at the tail end of syslog on the mounted SD-card from the problem Pi:

Code: Select all

pi@rpi3-agissr: $ tail -n 10 /mnt/usbroot/var/log/syslog
Oct  2 09:41:14 rpi3-agivpn dhcpcd[424]: eth0: probing address 10.0.0.235/24
Oct  2 09:41:14 rpi3-agivpn dhcpcd[424]: eth0: hardware address 00:40:8c:7c:a8:ef claims 10.0.0.235
Oct  2 09:41:15 rpi3-agivpn dhcpcd[424]: eth0: soliciting a DHCP lease
Oct  2 09:41:15 rpi3-agivpn dhcpcd[424]: eth0: offered 10.0.0.235 from 10.0.0.3
Oct  2 09:41:15 rpi3-agivpn dhcpcd[424]: eth0: probing address 10.0.0.235/24
Oct  2 09:41:15 rpi3-agivpn dhcpcd[424]: eth0: hardware address 00:40:8c:7c:a8:ef claims 10.0.0.235
Oct  2 09:41:16 rpi3-agivpn dhcpcd[424]: eth0: soliciting a DHCP lease
Oct  2 09:41:16 rpi3-agivpn dhcpcd[424]: eth0: offered 10.0.0.235 from 10.0.0.3
Oct  2 09:41:16 rpi3-agivpn dhcpcd[424]: eth0: probing address 10.0.0.235/24
Oct  2 09:41:16 rpi3-agivpn dhcpcd[424]: eth0: hardware address 00:40:8c:7c:a8:ef claims 10.0.0.235
If I go back in the log it is filled up with these lines...

What could be causing this to happen? The RPi should run on address 235 and this is what is being offered by DHCP, so why is it not accepting the address?
Can I do something on the card by editing a conf file somewhere to make it stop this loop?
Bo Berglund
Sweden

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Sat Oct 03, 2020 12:12 pm

Bosse_B wrote:
Sat Oct 03, 2020 8:51 am
What could be causing this to happen? The RPi should run on address 235 and this is what is being offered by DHCP, so why is it not accepting the address?
Can I do something on the card by editing a conf file somewhere to make it stop this loop?
Turns out that the MAC address in this log line:

Code: Select all

hardware address 00:40:8c:7c:a8:ef claims 10.0.0.235
is NOT an RPi MAC!!!
Some other unit on the network has stolen the IP address for the OpenVPN server!!! :evil: :twisted:
So when I thought I pinged and tried to SSH to the RPi it was in reality a completely different device!
Bo Berglund
Sweden

User avatar
FTrevorGowen
Forum Moderator
Forum Moderator
Posts: 5890
Joined: Mon Mar 04, 2013 6:12 pm
Location: Bristol, U.K.
Contact: Website

Re: Imaging an RPi SDcard on another RPi, how?

Sat Oct 03, 2020 1:29 pm

Bosse_B wrote:
Sat Oct 03, 2020 12:12 pm
Bosse_B wrote:
Sat Oct 03, 2020 8:51 am
What could be causing this to happen? The RPi should run on address 235 and this is what is being offered by DHCP, so why is it not accepting the address?
Can I do something on the card by editing a conf file somewhere to make it stop this loop?
Turns out that the MAC address in this log line:

Code: Select all

hardware address 00:40:8c:7c:a8:ef claims 10.0.0.235
is NOT an RPi MAC!!!
Some other unit on the network has stolen the IP address for the OpenVPN server!!! :evil: :twisted:
So when I thought I pinged and tried to SSH to the RPi it was in reality a completely different device!
Has someone changed (or "fiddled" with) the router the Pi was connected to? Was the Pi setup with a static IP address which was outside the range the router assigned by DHCP (and that router assigned range changed)? The "best way" (where possible) rather than use any static address on the Pi, or rely only on DNS, AIUI, is to reserve an address on the router, tied to the Pi's MAC address (which should not change) and should not be "handed out" by the router to something else. Although none of my Pi's are as "remote" as yours, I've used that method for many years with different routers and for most, if not all, of other devices expected to need a "stable" connection.. However it does rely on someone trusted with admin access rights to the router to "do things properly" and I appreciate that may be difficult in your circumstances.
Trev.
Still running Raspbian Jessie or Stretch on some older Pi's (an A, B1, 2xB2, B+, P2B, 3xP0, P0W, 2xP3A+, P3B+, P3B, B+, and a A+) but Buster on the P4B's. See: https://www.cpmspectrepi.uk/raspberry_pi/raspiidx.htm

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Sat Oct 03, 2020 3:44 pm

In actual fact the DHCP service is on our Active Directory Domain Controller and I have set a reservation there for the RPi3. Also I have configured the RPi3 to use that address as a static address so that it will operate even if the domain controller should go down. The "router" is in reality a Fortigate Firewall which does not serve as a DHCP server AFAIK.
But it has 2 port forward rules for OpenVPN targeting the RPi3 on address 235.

Now I have looked up the MAC address maker and it turns out to be AXIS, which is a security camera maker.
So this smells like someone installing a security system and pulling a static IP for the camera out of the range outside the DHCP handout range. But without checking the reservations on the domain controller...

Will investigate more next week, everybody are off over the week-end.
Bo Berglund
Sweden

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Sat Oct 03, 2020 7:43 pm

Turns out that the mac address was for a very old AXIS Print Server!
And its config web page could be accessed so I could switch it from static address to DHCP.
Now we also need to find it physically so we can scrap it.
Bo Berglund
Sweden

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Sun Oct 04, 2020 8:01 am

Just related questions concerning img files:
- Can an image file created by dd be mounted as a drive on RPi4?
- If so can the img file be located on an nfs network share?
Finally:
- If the above is possible, can I alter the so mounted image file by editing files on it?

Of course it would be good to not only get yes/no replies but actual advice on HOW to do it...

Or should I start another thread for these questions?

EDIT:
I found this excellent forum thread describing what I asked for! :D
So no need for further search for answers...
Bo Berglund
Sweden

nliviu
Posts: 51
Joined: Tue Jun 23, 2020 1:24 pm
Location: Romania

Re: Imaging an RPi SDcard on another RPi, how?

Sun Oct 04, 2020 2:22 pm

FYI, imgclone supports cloning another device then the currently mounted sd card:

Code: Select all

$ ./imgclone -?
----    Raspberry Pi clone to image V1.8    ---
-----------------------------------------------
---- DO NOT CHANGE FILES ON YOUR SD CARD    ---
---- WHILE THE BACKUP PROGRAM IS RUNNING    ---
---- THE DESTINATION .IMG FILE MUST BE      ---
---- ON AN EXTERNAL STORAGE / NETWORK SHARE ---
-----------------------------------------------
Create a backup of your SD card to an image file.
Warning! the <destination_file> image file must be located on an external drive, you cannot backup to a file on the SD card!
Usage:
imgclone [-u 1] [-s <source_device>] -d <destination_file>
	-u 1				   optional creates new UUID for the partitions.
	-s <source_device>     creates a backup of source_device, optional default is /dev/mmcblk0.
	-d <destination_file>  backup to destination_file.
    -x <number>            add <number> extra bytes of free space to the last partition.
    -p			           write copy progress to output.
    -bzip2  	           use bzip2 command to compress the image after cloning.
    -gzip   	           use gzip  command to compress the image after cloning.

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Imaging an RPi SDcard on another RPi, how?

Sun Oct 04, 2020 3:33 pm

PERFECT!
Had I know this yesterday I could have saved myself a good bit of trouble...
I have now added this to my evolving documentation on how to do stuff in Linux on an RPi!
Thanks for pointing it out! :) :D
Bo Berglund
Sweden

Return to “General discussion”