dd takes so long...

Hello,

I use dd to make a back up of my 16gb sd card. It does the job ok but it takes a such long time to complete.

Am i using the correct dd command in OS X to get the job done swiftly or is 6.6 hours typical for a 16 gb sd card to be dd'd to a .img?
My command and the final out put are below for reference.

Code: Select all

$ sudo dd bs=1m if=/dev/rdisk1 of=~/Desktop/ssbackup04.img

Code: Select all

15193+1 records in
15193+1 records out
15931539456 bytes transferred in 23842.935850 secs (668187 bytes/sec)


Thanks for any advice.

Have you tried increasing the block size ? I use bs=32M.
PeterO

The dd command is fine, but obviously your read speed is awful, suggesting a hardware issue on the Mac. Can you try a different SD-to-USB adapter/card reader, maybe there's a problem with your current one.

SD card manufacturer and model name would be helpful to help you help us help you

Have you tried increasing the block size ? I use bs=32M.
PeterO

I haven't, not too sure what the bs actually did in real terms. I have seen different dd how to guides using bs=2 and bs=1.

What am i telling bs to do by changing block size to say 32?

SD card manufacturer and model name would be helpful to help you help us help you

Is a Sandisk class 6 ultra 16gb card, it works well sofar with day to day Pi tinkering.

The dd command is fine, but obviously your read speed is awful, suggesting a hardware issue on the Mac. Can you try a different SD-to-USB adapter/card reader, maybe there's a problem with your current one.

I dont have another SD card reader to try that isnt SDHC compatible, it seems to be ok in other uses. The macbook generally runs very well despite being 6 years old, I cant pin it to be a hardware issue but that said I'm no expert

My sandisk ultra class 4 16gb SD card took only about 30 minutes or so on my pretty old macbook. Writing 2 gb image to the SD card took 10min and backing 16gb image took 30-40 min. Your Macbook should be fine and SD card should be fine. Probably your card reader. Dynex reader butchered my microsd useless, address mapping got bitched up and I ain't using dynex no mo its bad to assume the reader was wrong but I'm 90 percent sure, it was flaky reader before that problem.

I haven't, not too sure what the bs actually did in real terms. I have seen different dd how to guides using bs=2 and bs=1.
What am i telling bs to do by changing block size to say 32?

With bs=1 dd will read and write one byte at time thus incurring huge OS call overheads.
With bs=32M dd will read and write 32M at a time, thus incurring 1/32M of OS call overheads.
On sata hard disks this certainly makes a difference, but on slow devices like SDcards the improvement is less as the speed is dominated by the card access, but ISTR it did make a useful improvement the last time I timed a dd.
PeterO

With bs=1 dd will read and write one byte at time thus incurring huge OS call overheads.
With bs=32M dd will read and write 32M at a time, thus incurring 1/32M of OS call overheads.
On sata hard disks this certainly makes a difference, but on slow devices like SDcards the improvement is less as the speed is dominated by the card access, but ISTR it did make a useful improvement the last time I timed a dd.
PeterO

Hello Peter

At the risk of sounding a bit thick, this suggestion still writes the whole image file presumably?

Can I get data loss/damage or degradation from higher block sizes.

Thanks

Yes it still writes the whole image, no you won't get data loss/damage or degradation from higher block sizes.

I checked the manpage for dd and the command that you used is not valid
$ sudo dd bs=1m if=/dev/rdisk1 of=~/Desktop/ssbackup04.img

If you want to transfer 1Megabyte at a time you should use bs=1M instead of bs=1m.

There is a good chance that dd is ignoring the lowercase m (which is not valid). This means that it is reading and writing one byte at a time, instead of one Megabyte at a time.

I checked the manpage for dd and the command that you used is not valid
$ sudo dd bs=1m if=/dev/rdisk1 of=~/Desktop/ssbackup04.img

If you want to transfer 1Megabyte at a time you should use bs=1M instead of bs=1m.

There is a good chance that dd is ignoring the lowercase m (which is not valid). This means that it is reading and writing one byte at a time, instead of one Megabyte at a time.

Except that if you look at the record counts that the OP posted it is clear that it is NOT transferring single bytes.
PeterO

Except that if you look at the record counts that the OP posted it is clear that it is NOT transferring single bytes.
PeterO

Oh dear, it was a good theory for a short while.

Hi! I tried to install the .img of raspbian on SD card (32Gb class 10). But dd did not mounted the swap partition. Why this happened?

:?: Hi! I tried to install the .img of raspbian on SD card (32Gb class 10). But dd did not mounted the swap partition. Why this happened?

dd does not mount file systems, it just copies contents of the img, as far as I know. What exactly did you do? Post the code and output if possible.

Also post details about your card, manufacturer and model, in case that is an issue. Is it on the Verified Peripherals page?

http://elinux.org/RPi_VerifiedPeripherals

rsync (525 kb) from the debian wheezy main repository can transfer a .img file with lightning fast speed. See the script at https://gist.github.com/1004950

:?: Hi! I tried to install the .img of raspbian on SD card (32Gb class 10). But dd did not mounted the swap partition. Why this happened?

dd does not mount file systems, it just copies contents of the img, as far as I know. What exactly did you do? Post the code and output if possible.

Also post details about your card, manufacturer and model, in case that is an issue. Is it on the Verified Peripherals page?

http://elinux.org/RPi_VerifiedPeripherals

I'm just used this command "dd if=/dev/zero of=/dev/sda bs=1M". Are there script for mount perfect system partition?

:?: Hi! I tried to install the .img of raspbian on SD card (32Gb class 10). But dd did not mounted the swap partition. Why this happened?

dd does not mount file systems, it just copies contents of the img, as far as I know. What exactly did you do? Post the code and output if possible.

Also post details about your card, manufacturer and model, in case that is an issue. Is it on the Verified Peripherals page?

http://elinux.org/RPi_VerifiedPeripherals

I'm just used this command "dd if=/dev/zero of=/dev/sda bs=1M". Are there script for mount perfect system partition?

Err ...
that should simply delete your whole SD card.
If you want to write a image to your SD card try this :

Code: Select all

dd bs=32M if=~/Desktop/IMAGENAME.img of=/dev/DEVICE

Don't forget to change IMAGENAME and DEVICE according to your setup.

ghans

Sounds like the card reader is working in USB 1.1 compatible mode.

Can you format your SD Card to fat/fat32 and copy a large file, does it also take a while to process??

Richard S.

CTRL+t reports the progress of the dd image copy operation.

Old topic, but I'll add this anyway...

Sadly it seems that no one has picked up on the fact that the OP is dd to copy data OFF of a 16GB device. Therefore it is moving all 16GB even the unused space. This is very different from using dd to put an raspbian image ONTO a 16GB image.

Consider doing this on a device that has identical read and write characteristics (which none do, but...):

1. use dd to place 2013-02-09-wheezy-raspbian.img onto a 16GB device
2. use raspi-config to resizefs to fill the disk
3. use dd to make a backup of the device

In this simple senario you copy
Step 1 moves 1.8GB of data and takes about 20 minutes
Step 3 moves 16GB of data and takes about 180 minutes

This happens even though there is only 1.3GB of data on the device (as reported by du -sch).

The tool dd is not ideal for this. I suggest using bzipped tar. Full SD recovery becomes a little more complicated because you will want to reimage the device from the original image and the use tar to replace the contents on the data partition. However, it is very simple to extract a subset of files from a tar. So, it's a tradeoff.