Page 1 of 1
Failing to copy files to USB
Posted: Tue Sep 30, 2014 10:39 am
by ppparadox
Hi, i'm having issues with my raspberry pi model B (running a fully up-to-date raspbian) and my 8GB USB storage device: i am developing a small embedded system which has to check for the presence of certain files inside the USB device and copy them (if they're not there). My problem is that only few (3-5) out of ~20 files in dir1 are actually being copied to USB. What can i do?
It all happens via bash script (this are just the last few lines):
Code: Select all
if [[ ! -d "$STORAGE"/dir1 ]]; then
cp -rf "$SOURCE"/dir1 "$STORAGE"
fi
sleep 10
sync
sleep 20
sync
shutdown -h now
#lsusb -v output:
Code: Select all
Bus 001 Device 005: ID 0718:0619 Imation Corp.
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x0718 Imation Corp.
idProduct 0x0619
bcdDevice 1.00
iManufacturer 1 Imation
iProduct 2 Nano Pro
iSerial 3 [redacted]
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 32
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 100mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 3:48 pm
by cpc464
Well what error do you see when running the script ? Also, is there any particular reason for using the -f flag on cp.
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 4:13 pm
by ppparadox
No errors, the -f flag is just to overwrite without prompting.
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 4:31 pm
by cpc464
Couple of things. The "-f" flag is not for overwriting I think. cp should overwrite by default. However, with your code, the issue of overwriting should never arise, since you check that $STORAGE/dir1 does not exist before newly creating $STORAGE"/dir1 with the cp command.
Try using the "-v" flag with cp to see what is happening. And maybe put the "-x" flag at the start of your script, to see commands a they are run.
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 4:57 pm
by ppparadox
Been there, done that. No problem whatsoever. Also as i said the problem is that some files get copied, not all of them, which smells like some ugly form of caching is in the way.
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 5:12 pm
by ilguargua
I would try to umount the usb device before shutdown, maybe can help, even if the system should properly umount all devices before shutting down...
Cheers, Ale
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 6:00 pm
by cpc464
I am not sure what you mean by "been there, done that". Disk caching is unlikely to be the culprit. If cp is failing to copy files which you think it should be copying, cp -v, #!/bin/bash -x will show why. dmesg or tail /var/log/messages or syslog will show any hardware errors.
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 6:53 pm
by ppparadox
Ok i did some more testing and now i think i've narrowed down the problem: if i run the script without the final "shutdown -h now" everything is OK.
Still it is very worrying that i'm losing data even after TWO calls to sync and a proper shutdown. How come?
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 7:03 pm
by DougieLawson
What's the filesystem on the USB stick? Have you tried explicitly umounting it (to ensure it's buffers are flushed)? Have you considered reformatting it to ext4?
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 7:09 pm
by ppparadox
File system is FAT32, unmounting before shutdown is the only thing i haven't tried yet. Reformatting is out of the question because of the nature of what i'm trying to accomplish (windows compatibility yay). But is there that big a difference from calling sync?
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 7:35 pm
by ilguargua
Please try, and le us know the answer!
Cheers, Ale.
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 7:54 pm
by DougieLawson
ppparadox wrote:File system is FAT32, unmounting before shutdown is the only thing i haven't tried yet. Reformatting is out of the question because of the nature of what i'm trying to accomplish (windows compatibility yay). But is there that big a difference from calling sync?
https://bugs.debian.org/cgi-bin/bugrepo ... bug=579575
suggests using
mount -t vfat -o flush,other_options_here ... to force a flush when the copy is complete.
Re: Failing to copy files to USB
Posted: Tue Sep 30, 2014 8:25 pm
by ppparadox
I managed to solve the issue. The situation was the following, a bit more complicated than i initially said:
/etc/rc.local contained calls to several scripts.
The first one was meant for the
first boot only and after a few things had to shutdown but the very next script was the one containing the problematic file checking. So after the first script had successfully exited (after calling shutdown) the next one had just started copying those files but didn't have enough time to finish before being terminated by the pending shutdown.
Upon rebooting the second script would have been called again (not the first, that was for the first boot only) but the folder was there as far as if [[ ! -d "$folder" ]] was concerned... just not all of the files inside.
Maybe it would have been better to use rsync and check for each and every file.
Still worrying though. I thought file copying was more reliable than this.
Thanks to all of you for your kind answers!