obieq
Posts: 6
Joined: Wed Jan 11, 2017 7:54 pm

Rpi3 + Usb + Samba - problem with copying large files

Wed Jan 11, 2017 8:36 pm

Hello,
I am using Raspberry Pi3 with mounted external usb 500gb disk formatted with ntfs (sometimes I need to unplug disk and attach it directly to PC to copy larger volume of data).
Rpi3 is connected to router with cable and disk has its own power supply.
Rpi3 usb disk is LAN shared with Samba.
I can access network share with Windows 10 i can read and copy files i also mapped it.
But when I am trying to copy big files like Movie (500-1400mb) from Windows network folder/mapped disk (no matter).
Windows stucks on preparing, caluclating time. After several seconds I am getting error 0x8007003B - an unexpected network error - after that network share disappears for about 30-60 seconds.

I have checked samba logs but there's nothing interesting.

300cpilot
Posts: 7
Joined: Sat Jan 07, 2017 5:07 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Fri Jan 13, 2017 5:34 am

I am curious if because the nic is on the usb bus and the drive is on it as well? If you are not simply running out of buffer.

obieq
Posts: 6
Joined: Wed Jan 11, 2017 7:54 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Fri Jan 13, 2017 11:53 am

I dont understand can you explain ?
Should I use wifi instead of rj45 cable ?
Then most of RPi NAS (which is not the best solution) should disappear.
(that would be bizarre...)

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Rpi3 + Usb + Samba - problem with copying large files

Fri Jan 13, 2017 1:43 pm

You must have a special problem. I'm doing that all the time, transferring files in the Gigabyte range - between RPis, Windows (7) and Ubuntu Desktop.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

obieq
Posts: 6
Joined: Wed Jan 11, 2017 7:54 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jan 23, 2017 7:35 am

I have checked logs and there's message like.
OUT of memory killed process smb.

RPi is running only emby-server and kodi.
I dont wanna add more swap memory beacuse this will ruin my sd card.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jan 23, 2017 8:24 am

I just copied a 6.7 GB file across Samba (one RPi to another) and noticed no increase of memory usage at all.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

obieq
Posts: 6
Joined: Wed Jan 11, 2017 7:54 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jan 23, 2017 8:32 am

I am amble copy small files up to 100-300 mb this to attached USB 500GB disk with NTFS (maybe NTFS eats out all memory?)
But when I try copy a movie 600mb+ there's long "calculating" period then samba is killed and dissapear from network, also my putty ssh connection freezes.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jan 23, 2017 8:55 am

obieq wrote:I am amble copy small files up to 100-300 mb this to attached USB 500GB disk with NTFS (maybe NTFS eats out all memory?)
But when I try copy a movie 600mb+ there's long "calculating" period then samba is killed and dissapear from network, also my putty ssh connection freezes.
Run "top" in your putty ssh window and watch what's happening.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

hortimech
Posts: 444
Joined: Wed Apr 08, 2015 5:52 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jan 23, 2017 9:29 am

Are you connecting to the share by its short hostname or by its FQDN ?
If you are doing something like '\\fileserver\share', try '\\fileserver.domain.tld' instead.

obieq
Posts: 6
Joined: Wed Jan 11, 2017 7:54 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jan 23, 2017 8:49 pm

I have tried
\\192.168.1.110\sharename
\\raspberrypi.local\sharename
both with same result

demsg / kern.log
http://pastebin.com/uQ2bXx8x

I not found any interesting logs regarding samba etc.

Htop and copy window screen.
Image

Seems like emby process multiplicates something many times while copying and leech all available memory

hortimech
Posts: 444
Joined: Wed Apr 08, 2015 5:52 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jan 23, 2017 9:07 pm

I see that using '.local' for the dns domain isn't a good idea hasn't filtered on to this mailing list yet ;)

Using '.local' interferes with Avahi (and vica-versa)

Can you post your smb.conf


hortimech
Posts: 444
Joined: Wed Apr 08, 2015 5:52 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Tue Jan 24, 2017 10:16 am

OK, first, next time remove all the commented lines before posting your smb.conf :)

Is the win10 computer part of the workgroup 'WORKGROUP' ?
Does the user 'obieq' exist on the win10 computer and as a Unix and Samba user on the Samba machine, all with the same password ?
Does the group 'obieq' exist on the Samba machine ?

Are the 'smbd' and 'nmbd' binaries running on the Samba machine ?

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Rpi3 + Usb + Samba - problem with copying large files

Tue Jan 24, 2017 11:37 am

This doesn't look like a Samba problem at all but as a (memory) problem caused by the emby-server. I have no experience with this media server but I suppose this feature
Automatically sync new content as you add it.
runs amok when you are adding large movie files which are not completely transferred yet.
The only solution may be to stop the server while transferring files.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

RafaelP
Posts: 2
Joined: Sat May 05, 2018 8:13 am

Re: Rpi3 + Usb + Samba - problem with copying large files

Sun Jun 03, 2018 2:11 am

Could you fix the problem? My problem is very similar:
After rebooting I can transfer files through sftp (using filezilla on windows) without issue but after half an hour or so after booting up the transfers don't work anymore: It hangs after a second or two of high speed transfer, then times out after and tries to reconnect after 20 seconds only to fail again after a second or two of transfer. I also tried the same with transferring the files from the sd card (without hdd connected) and it has the same issues.

https://github.com/raspberrypi/linux/issues/2449

seems like a RPI3B+ problem

michen
Posts: 2
Joined: Mon Jun 25, 2018 5:49 am

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jun 25, 2018 5:51 am

Same here! Read somewhere it might have to do with underlying FAT32 filesystem, might be worth a shot to try a linux native flavor.

IngoKnito
Posts: 12
Joined: Mon Jun 25, 2018 8:20 am

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jun 25, 2018 8:21 am

Same here :D - with a little difference: Same issue on a PI1!

LTolledo
Posts: 4151
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jun 25, 2018 11:19 am

Ok here's my test setup:

A. Raspberry Pi B+ (lets call it RPi1B+) , with
A1. Raspbian Stretch with Desktop on 16GB microsSD,
A2. Ethernet connection to Ethernet Switch HUB -B
A3. Samba server and client installed, running fine
A4. 1 share folder in Ext4 format (inside microsd)
A5. 1 share folder in NTFS formatted 128GB USB flash memory
A6. Deluge running in the background, 6 in queue, downloading 2 torrent files 400MB each, saving to 128GB USB flash
A7. of the 512MB on board memory, 192MB is allocated for video because of Kodi
A8. RPi1B+ currently running headless (despite the setup above)
A9. RP1B+ running non-stop in the last 36Hrs prior to this test

B. NAS (EXT4 format) connected to Ethernet Switch HUB- A

C. Windows10 PC connected to Ethernet Switch HUB- A
C1. RPi1B+'s NTFS samba share mapped into windows
C2. NAS folder is mapped into windows

D. Ethernet Switch HUB -B connected to Ethernet Switch Hub -A

What I did:

Using Windows10 PC, copied an 8.7GB movie file from NAS directly to NTFS share folder in RPi1B+

Finished copying the 8.7GB file, took a little over an hour. No errors encountered during file transfer.
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27442
Joined: Sat Jul 30, 2011 7:41 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Mon Jun 25, 2018 1:08 pm

This thread is getting a little confusing with reports from almost all models.

Can those people who can replicate the issue at will, indicate their exact issues along with: Model of Pi, version of OS (uname -a), type of file system used on USB drive (e.g. NTFS), destination system type, network type. Also, anything non-standard attached to the Pi, or to the network, would also help. If anyone has dmesg logs showing errors, that would also be very useful.

My testing shows no faults on anything at the moment. We did have problem on NTFS attached USBdrive on the 3b+ but this is fixed in latest rpi-update. This was related to the tcp-segmentation offloading on the gigabit lan chip.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

ejolson
Posts: 6027
Joined: Tue Mar 18, 2014 11:47 am

Re: Rpi3 + Usb + Samba - problem with copying large files

Wed Jun 27, 2018 1:19 am

jamesh wrote:
Mon Jun 25, 2018 1:08 pm
We did have problem on NTFS attached USBdrive on the 3b+ but this is fixed in latest rpi-update. This was related to the tcp-segmentation offloading on the gigabit lan chip.
I reported a similar issue with a USB attached ext4 filesystem on a thumb drive exported over NFS. Turning off tcp-segmentation offloading allows it to work. Will

# apt-get update
# apt-get upgrade

pull in the fixed driver? Does the fixed driver have any advantages over entering the configuration command

# ethtool -K eth0 tx-tcp-segmentation off

and using the original driver?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27442
Joined: Sat Jul 30, 2011 7:41 pm

Re: Rpi3 + Usb + Samba - problem with copying large files

Wed Jun 27, 2018 8:49 am

ejolson wrote:
Wed Jun 27, 2018 1:19 am
jamesh wrote:
Mon Jun 25, 2018 1:08 pm
We did have problem on NTFS attached USBdrive on the 3b+ but this is fixed in latest rpi-update. This was related to the tcp-segmentation offloading on the gigabit lan chip.
I reported a similar issue with a USB attached ext4 filesystem on a thumb drive exported over NFS. Turning off tcp-segmentation offloading allows it to work. Will

# apt-get update
# apt-get upgrade

pull in the fixed driver? Does the fixed driver have any advantages over entering the configuration command

# ethtool -K eth0 tx-tcp-segmentation off

and using the original driver?
Not sure if the fix has made it to the repo yet. Definitely in rpi-update. The fix is to simply disable the segmentation in the driver at startup
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

michen
Posts: 2
Joined: Mon Jun 25, 2018 5:49 am

Re: Rpi3 + Usb + Samba - problem with copying large files

Sat Jun 30, 2018 5:49 am

Hooray. :D :D The newest update has resolved the issue. So, an apt-get update and apt-get upgrade plus a simple reboot fixed it for me.

IngoKnito
Posts: 12
Joined: Mon Jun 25, 2018 8:20 am

Re: Rpi3 + Usb + Samba - problem with copying large files

Sun Jul 22, 2018 6:20 am

* Raspi 1
* Linux raspberrypi 4.14.34+ #1110 Mon Apr 16 14:51:42 BST 2018 armv6l GNU/Linux
* File system: FAT
* Shared with Samba
* Nothing else is attached to the PI

The Problem occurs, when the files are greater around 80 MByte.

Filetransfer over Samba: Error "Unexpected Network-Error"
Filetransfer over FTP: Works fine!
Shorter Files were transfered fine.

I have done the 'apt-get update' and 'apt-get upgrade'

Additional Info: A filetransfer with 100 MByte over Samba to an internal folder (\\raspberrypi\pi\Public) works fine. The same file to the attached USB-Device not.
Last edited by IngoKnito on Sun Jul 22, 2018 6:27 am, edited 1 time in total.

DirkS
Posts: 10452
Joined: Tue Jun 19, 2012 9:46 pm
Location: Essex, UK

Re: Rpi3 + Usb + Samba - problem with copying large files

Sun Jul 22, 2018 6:26 am

IngoKnito wrote:
Sun Jul 22, 2018 6:20 am
I have done the 'apt-get update' and 'apt-get upgrade'
If you have standard Raspbian Stretch installed and it's fully updated you should be on 4.14.52, not 4.14.34
Try to update / upgrade again.
If nothing gets updated you need to tell us exactly which OS + version you're running,
Also model Pi, if you're using wifi or wired, etc.

IngoKnito
Posts: 12
Joined: Mon Jun 25, 2018 8:20 am

Re: Rpi3 + Usb + Samba - problem with copying large files

Sun Jul 22, 2018 6:37 am

First of all: Thanks for the fast help! :-)

Yes, i have installed the standard OS and have done update/upgrade again: No change!

-----------------
pi@raspberrypi:~ $ sudo apt-get update
Hit:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Hit:2 http://archive.raspberrypi.org/debian stretch InRelease
Reading package lists... Done
pi@raspberrypi:~ $ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
chromium-browser nodejs nodered rpi-chromium-mods sense-emu-tools
0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.14.34+ #1110 Mon Apr 16 14:51:42 BST 2018 armv6l GNU/Linux
-----------------

The model has no WIFI, so connected via wired LAN.

How can i find out the exact version rahter than 'uname -a'?

Return to “Troubleshooting”