Roaders
Posts: 13
Joined: Sun Mar 20, 2016 8:44 am

How to automatically deal with usb drives added and removed

Mon Jul 13, 2020 2:55 pm

Hi All

I want to setup a raspberry pi that I can leave powered on long term in my garage. I want to plug in usb drives and have the contents automatically moved to network storage.

Reading the external storage documentation it seems that I need to setup each usb drive that I want to use ahead of time in the fstab file? Is that really the case?

The documentation says that I need to unmount the drive as well. Is this really the case? You're supposed to unmount on windows and I basically never do. I know that if it's in the middle of a copy things will go bad but if the move is complete and there are no files on the drive anymore will it be ok to just pull it out without ummounting?

Thanks

pidd
Posts: 937
Joined: Fri May 29, 2020 8:29 pm
Location: Birkenhead, Wirral, UK
Contact: Website

Re: How to automatically deal with usb drives added and removed

Mon Jul 13, 2020 3:08 pm

You need to umount, you might get away with it for years but at some point in time it will bite you when you least want it to. We've found instances where data was corrupted and the corruption gone unnoticed for months, sometimes the integrity of the data is more important than others.

The transfer software can easily umount automatically once it has done the transfer so it shouldn't be too much of an issue.

User avatar
RaspbianUser1
Posts: 300
Joined: Thu Mar 05, 2020 6:34 pm

Re: How to automatically deal with usb drives added and removed

Mon Jul 13, 2020 3:09 pm

Roaders wrote:
Mon Jul 13, 2020 2:55 pm
Hi All

I want to setup a raspberry pi that I can leave powered on long term in my garage. I want to plug in usb drives and have the contents automatically moved to network storage.

Reading the external storage documentation it seems that I need to setup each usb drive that I want to use ahead of time in the fstab file? Is that really the case?

The documentation says that I need to unmount the drive as well. Is this really the case? You're supposed to unmount on windows and I basically never do. I know that if it's in the middle of a copy things will go bad but if the move is complete and there are no files on the drive anymore will it be ok to just pull it out without ummounting?

Thanks
To answer the unmounting question, its up to you, run a command or even set up a alias for it and save all of your data or live life on the dangerous side and risk it all to the gods of corruption, if its not too much of a pain, just make a .bash_aliases file and make an alias to make the unmount command smaller.

Small effort = No risk
no effort = risk :mrgreen:
Running Raspberry Pi OS Full, therefore my comments will be based on that
Using RPI-4B 4GB at 2.1GHz CPU and 700MHz GPU
don't format your sd because someone said so.
Or put your favourite cutlery into your nearest outlet (May result in death) ;)

Roaders
Posts: 13
Joined: Sun Mar 20, 2016 8:44 am

Re: How to automatically deal with usb drives added and removed

Mon Jul 13, 2020 3:56 pm

Thanks for the replies.

Just to clarify when you say "you might get away with it", what is the worst that can happen if I don't get away with it? At the end of the copy process all files will be deleted so I can't see what could be corrupted.

I'll implement unmounting, hopefully won't be too much of an issue.

pidd
Posts: 937
Joined: Fri May 29, 2020 8:29 pm
Location: Birkenhead, Wirral, UK
Contact: Website

Re: How to automatically deal with usb drives added and removed

Mon Jul 13, 2020 4:10 pm

The worst case is the least likely and that is damage to the device, while they are designed to be foolproof there are sometimes some very small time windows where they can get corrupted in a manner that the device itself can't correct - but it would take a lot of coincidences for this to happen.

More usually you simply lose a file but sometimes the length of a file is incorrect however another case that is more indiscernible is a small number of bytes corrupted within the file, you might find some programs will still read the file and others won't etc or data simply incorrect.

If the format of the device is corrupted the problem might arise at a later date when it attempts to use that section - I think this is less likely these days as the management is done within the device itself.

The problem is that yanking it out can create a variety of random events, not a predictable outcome.

DarkElvenAngel
Posts: 941
Joined: Tue Mar 20, 2018 9:53 pm

Re: How to automatically deal with usb drives added and removed

Mon Jul 13, 2020 4:34 pm

Roaders wrote:
Mon Jul 13, 2020 2:55 pm
Hi All

I want to setup a raspberry pi that I can leave powered on long term in my garage. I want to plug in usb drives and have the contents automatically moved to network storage.

Reading the external storage documentation it seems that I need to setup each usb drive that I want to use ahead of time in the fstab file? Is that really the case?

The documentation says that I need to unmount the drive as well. Is this really the case? You're supposed to unmount on windows and I basically never do. I know that if it's in the middle of a copy things will go bad but if the move is complete and there are no files on the drive anymore will it be ok to just pull it out without ummounting?

Thanks
Are you running this pi headless?

Am I understanding what you want to do is plug the USB in have it mount automatically then copy it's contents off then make it safe to remove the usb when it's finished?

If so you can write a script to do this I would suggest adding an LED or some indication that the drive is safe to pull out again.

You can do the same with a GUI interface but it's a bit more complex.

Roaders
Posts: 13
Joined: Sun Mar 20, 2016 8:44 am

Re: How to automatically deal with usb drives added and removed

Tue Jul 14, 2020 4:29 am

Yes, I am considering an led or e-ink display or something similar.

DarkElvenAngel
Posts: 941
Joined: Tue Mar 20, 2018 9:53 pm

Re: How to automatically deal with usb drives added and removed

Wed Jul 15, 2020 1:33 pm

Roaders wrote:
Tue Jul 14, 2020 4:29 am
Yes, I am considering an led or e-ink display or something similar.
Okay well seems like this post has the idea of the solution viewtopic.php?f=66&t=195948#p1228119 I think it can be modified to fit what you want to do.

Basically when it says, when the USB device is put in one part of the system called udev will detect it and this has to start another script to do the copying and make the drive safe to remove again.

I've done something similar with a kiosk, the USB stick gets plugged in and then copy off the updated contents.

Return to “General discussion”