Page 1 of 1

Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 10:43 am
by morphy_richards
I know there was a thread on this a while back but I cant seem to find it now ... Lots of info out there on copying a single image to a single card but ...

I've got a nice up to date working image of Raspbian that's set up with a few extras installed such as Geany copied to a hard disk.
I've copied this image successfully to another SD card using Win32 Disk Imager.

The problem is I've got fifteen SD Cards to copy.

Equipment I've got to hand is

a number of 10 port USB hubs
15 rev 2 Raspberry Pi's
Some Windows Computers (but only have admin access and therefore ability to run W32 Disk Imager on one computer.)

What's the best way of copying the image to 10 or 15 cards at once - (it's not a problem if it needs to run overnight)

Re: Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 11:05 am
by jamiesk

Re: Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 11:28 am
by morphy_richards
http://raspberrypi.stackexchange.com/qu ... o-many-sds
Given that no-one seems to have produced a multi-card SD card reader (one that can hold multiple SD cards at once) and the fact that that you can buy a USB hub and a bunch of single SD card USB readers very cheaply, one way to speed up the process would be to make your own SD card duplicator.
Okay ..

Theoretically ...
from http://raspberrypi.stackexchange.com/qu ... spberry-pi
Creating a disk image will preserve not only files but also the filesystem structure and when you decide to flash your new SD card, you will be able to just plug it in and it will work.

On Linux, you can use the standard dd tool:

dd if=/dev/sdx of=/path/to/image bs=1M
Where /dev/sdx is your SD card.
So , could I plug a load of SD Card readers & cards into a multi-port hub on a Raspberry Pi
Start one dd copy process going
Detach it - (using something like nohup? or Screen?)
Start the next one going...
Repeat process until all cards are being written
Go home for the weekend
Come back on Monday to 15 written SD cards?

Re: Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 12:26 pm
by morphy_richards
Thinking about this a bit more ...

I think it would be generally useful to have a Bash script that could automate the entire process. - to be shared with schools who need to setup lots of Pis in a batch job.
Assume You have a Raspberry Pi with a powered USB hub plugged in

The output might go something like this...

[email protected] ~ $ clonedisks
Insert the master disk into a USB port and press Enter >
Checking disk ... ok
Insert a blank disk for copying into a new USB port and press Enter >
...ok. Do you want to copy another y/n? > y
Insert a blank disk for copying into a new USB port and press Enter >
...ok. Do you want to copy another y/n? > y
Insert a blank disk for copying into a new USB port and press Enter >
...ok. Do you want to copy another y/n? > y
Insert a blank disk for copying into a new USB port and press Enter >
...ok. Do you want to copy another y/n? > n
Ready to clone disks. Press enter when ready>
Copying ...
...
...
...
Done.
[email protected] ~ $

Re: Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 12:49 pm
by jamiesk
A nice little script will do that for you :) If you mount all the devices then try ...

#!/bin/bash
for i in 1 2 3 4 5
do
echo "Starting to program $i"
dd if=/path/to/image of=/dev/sd$i bs=1M
echo "We have completed $i"
done

I have not tested this substitution of the loop variable $i in this ... maybe make a string of it and use "of=string"

My rPi is in the loft switching on and off the 1Kw hot water heater in this sun shine 8-)

Re: Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 12:53 pm
by morphy_richards
jamiesk wrote:A nice little script will do that for you :) If you mount all the devices then try ...
Excellent, thank you! I haven't made one of these since nineteen ninety something so tinkering with this will make a nice little Christmas holiday project...
jamiesk wrote:My rPi is in the loft switching on and off the 1Kw hot water heater in this sun shine 8-)
Lucky you! Most of the country here seems to now be underwater :D

Re: Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 2:38 pm
by morphy_richards
jamiesk wrote: #!/bin/bash
for i in 1 2 3 4 5
do
echo "Starting to program $i"
dd if=/path/to/image of=/dev/sd$i bs=1M
echo "We have completed $i"
done
This would do one after the other in a queue which is fine, however I'm wondering if it might be possible to make all the cards copy the from master in parallel /at the same time (not even sure if it would be possible like that with dd) by detaching the processes, and would that even make the job finish any faster.

Re: Copy image to multiple SD Cards.

Posted: Fri Dec 21, 2012 9:40 pm
by jamiesk
Yes you can. I would think it would go faster due to writing to a card is slower than reading.

Google found me http://superuser.com/questions/145516/c ... ltaneously which basically says

Code: Select all

cat drive.image | tee >(dd of=/dev/sda) >(dd of=/dev/sdb) >(dd of=/dev/sdc) | dd of=/dev/sdd
But how will you catch errors ? Or do you care?

I would put together files for each copy like (NOT TESTED)
#!/bin/bash
# for SD1
dd if=/path/to/image of=/dev/sd1 bs=1M > /tmp/sd1
mail -s "Sd1 done" [email protected] <~/email.done
and then call them all from a main one
#!/bin/bash
# to run all files from SD1 to SD5
./SD1 &&
./SD2 &&
./SD3 &&
./SD4 &&
./SD5 &&
And then each will send you an email of how it did. Assuming you have email setup!!

But you said it could be an overnighter, I would have hoped that 15 times 4G cards would be written a bit quicker than an overnighter!!!

Re: Copy image to multiple SD Cards.

Posted: Sat Dec 22, 2012 7:02 am
by morphy_richards
you said it could be an overnighter, I would have hoped that 15 times 4G cards would be written a bit quicker than an overnighter!!!
Originally I thought it could be but it would be good if we can make it go faster. If it could do a batch in a lunch break (doubtful but worth a go) that would be good.

Re: Copy image to multiple SD Cards.

Posted: Sat Dec 22, 2012 10:28 am
by jamiesk
Give it a go. Have one of the science kids with a stopwatch too :)

Also make sure the master has had the file expanded to the full size of the SD card too. You will be copying it, might as well make it right in the first place.

Re: Copy image to multiple SD Cards.

Posted: Sun Dec 30, 2012 9:37 am
by rpdom
Another way to copy to multiple cards simultaneously is to install and use the dcfldd package. It is like an enhanced version of dd with lots of useful extra options.

As an example (disclaimer - I haven't actually tired this as I don't have any spare cards right now), I'd use:

Code: Select all

dcfldd if=/path/to/image bs=1M sizeprobe=if of=/dev/sda of=/dev/sdb of=/dev/sdc (etc...)
As you can see, it supports multiple "of=" statements, and the "sizeprobe=if" means "Show a continuous progress report calculated using the size of the input file"

Re: Copy image to multiple SD Cards.

Posted: Fri Sep 13, 2013 11:36 am
by hagalin
Premise ;

A machine with running linux and root access
6 card readers/writers
6 copies of the image to be written

The operating system is running on /dev/sda and it is the only thing that is usally mounted

#!/bin/bash
# Unmount all media / disk to start clean
umount /media/*
UNIQ_DATE_ID=`date +%d%m%Y%H%M%S_$$`
# Find what disks are there
fdisk -l /dev/sd[qwertyuiopasdfghjklzxcvbnm]|grep dev|grep Disk|grep -v sda | awk '{print $2}' | awk -F"\:" '{print $1}'>>/tmp/disk_id$UNIQ_DATE_ID
# Generate disks varibles
DISK1=`cat /tmp/disk_id$UNIQ_DATE_ID | awk 'NR==1 {print}'`
DISK2=`cat /tmp/disk_id$UNIQ_DATE_ID | awk 'NR==2 {print}'`
DISK3=`cat /tmp/disk_id$UNIQ_DATE_ID | awk 'NR==3 {print}'`
DISK4=`cat /tmp/disk_id$UNIQ_DATE_ID | awk 'NR==4 {print}'`
DISK5=`cat /tmp/disk_id$UNIQ_DATE_ID | awk 'NR==5 {print}'`
DISK6=`cat /tmp/disk_id$UNIQ_DATE_ID | awk 'NR==6 {print}'`

# spit out to standard out disk devices
echo $DISK1
echo $DISK2
echo $DISK3
echo $DISK4
echo $DISK5
echo $DISK6

# start to copy take the image /img/some_img_a.img and send it to disk which is correct
dd if=/img/some_img.A.img of=$DISK1 bs=1048576 conv=noerror >> /tmp/dd_status-$UNIQ_DATE_ID &
dd if=/img/some_img.B.img of=$DISK2 bs=1048576 conv=noerror >> /tmp/dd_status-$UNIQ_DATE_ID &
dd if=/img/some_img.C.img of=$DISK3 bs=1048576 conv=noerror >> /tmp/dd_status-$UNIQ_DATE_ID &
dd if=/img/some_img.D.img of=$DISK4 bs=1048576 conv=noerror >> /tmp/dd_status-$UNIQ_DATE_ID &
dd if=/img/some_img.E.img of=$DISK5 bs=1048576 conv=noerror >> /tmp/dd_status-$UNIQ_DATE_ID &
dd if=/img/some_img.F.img of=$DISK6 bs=1048576 conv=noerror >> /tmp/dd_status-$UNIQ_DATE_ID &

echo `date` >> /tmp/dd_status-$UNIQ_DATE_ID

Re: Copy image to multiple SD Cards.

Posted: Sat Sep 14, 2013 2:15 pm
by hagalin
One caveat.

If no linux machine is there, you can always live boot a machine from for example ubuntu livecd to gain a temporary linux machine. After use, boot the machine normaly and it will be unchanged.
http://www.ubuntu.com/download/desktop

any linux would work as most if not all will have the tools used in the script.

Re: Copy image to multiple SD Cards.

Posted: Tue Oct 29, 2013 4:46 pm
by rockandscissor
We had a similar requirement for a project where we needed to be able to write images to 10+ Raspberry Pi SD cards on a daily basis. We looked at the various Card Duplicator's available to buy but found most of them won't successfully make a bootable SD card, as most only copy files directly and not the bootable information, they are also ridiculously expensive!

So as a solution we wrote our own simple web-based software to run on a single Raspberry Pi connected to 2 x 7-port Belkin powered USB hubs (The Pi has a limitation of 2 hubs, and a maximum of 14 USB ports).

We've now released this as Open Source software to help others who have the same requirement. Currently just the source code is available but we will also be uploading an image file based on Arch Linux ARM running Nginx + php

http://www.rockandscissor.com/projects/osid

Re: Copy image to multiple SD Cards.

Posted: Mon Nov 18, 2013 4:25 pm
by morphy_richards
rpdom wrote:Another way to copy to multiple cards simultaneously is to install and use the dcfldd package. It is like an enhanced version of dd with lots of useful extra options.

As an example (disclaimer - I haven't actually tired this as I don't have any spare cards right now), I'd use:

Code: Select all

dcfldd if=/path/to/image bs=1M sizeprobe=if of=/dev/sda of=/dev/sdb of=/dev/sdc (etc...)
As you can see, it supports multiple "of=" statements, and the "sizeprobe=if" means "Show a continuous progress report calculated using the size of the input file"
Giving this a go right now ...
fdisk -l to locate the mounted names of the sd cards and then the command as you gave it ... seems to be working, I have 5 cards being written (apparently, flashy lights going whizzy anyway) via a USB hub and 5 card reader/adapter thingies. Sadly it didn't seem able to detect any more than 5 even though the hub has 10 ports.
Image

Re: Copy image to multiple SD Cards.

Posted: Tue Nov 19, 2013 9:36 am
by morphy_richards
Success.

A batch of 5 cards plugged into a multi USB hub seem to take about 15 minutes to write.

Re: Copy image to multiple SD Cards.

Posted: Tue Dec 17, 2013 8:25 am
by morphy_richards
to copy the image from the working sd
dmesg | tail
use this to find the sd card
gives

Code: Select all

[80619.310254] sd 6:0:0:0: [sdb] Mode Sense: 03 00 00 00
[80619.310879] sd 6:0:0:0: [sdb] No Caching mode page found
[80619.310893] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[80619.316077] sd 6:0:0:0: [sdb] No Caching mode page found
[80619.316084] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[80619.317600]  sdb: sdb1 sdb2
[80619.321028] sd 6:0:0:0: [sdb] No Caching mode page found
[80619.321037] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[80619.321044] sd 6:0:0:0: [sdb] Attached SCSI removable disk
we want "[sdb] Attached SCSI removable disk"
So use dd to copy the file:
sudo -i
dd if=/dev/sdb of=~/filename.bin
This copies the file into the root home directory.
When it's finished, copy the file to somewhere more accessible and then issue the commands above to copy multiple images.

Re: Copy image to multiple SD Cards.

Posted: Tue Jan 14, 2014 10:12 pm
by morphy_richards
Word of warning. Pay attention when you list your sd cards using fdisk -l , I managed to accidentally over write my main linux partition on my ubuntu machine today with a copy of a rasbian image because I just assumed sdb would be a card. Doh!

Re: Copy image to multiple SD Cards.

Posted: Sun Jan 18, 2015 12:01 pm
by RomanB
Hi!

If you're on Windows, you can use this small utility https://github.com/RomanBelkov/DiskImager/releases to write on multiple SD cards.

I have tested it writing to 4 SD simultaneously, seems fine.

Re: Copy image to multiple SD Cards.

Posted: Thu Feb 19, 2015 7:57 pm
by topshed
It may be a little late now, but I wrote a simple tutorial for bulk-copying SD cards. http://richardhayler.blogspot.co.uk/201 ... -army.html

Re: Copy image to multiple SD Cards.

Posted: Thu Aug 03, 2017 9:39 am
by nayan8459
hey
I want to prepare mass sdcard with raspbian image, let say 100 for card within 30 minutes. Is there any idea to do the same..

Re: Copy image to multiple SD Cards.

Posted: Thu Aug 03, 2017 11:27 am
by DougieLawson