tgwena
Posts: 11
Joined: Tue Nov 06, 2012 12:30 am

NFS server limiting the number of connections. Why?

Sun Jun 24, 2018 3:24 am

I have built myself a cluster of 20 Raspberry machines. I have set one of them to be an NFS server, and all the others are NFS clients (including the server, for uniformity).

The /etc/exports configuration file for the server (named rg.local) has the line
/home/pi *(rw,sync,no_subtree_check)
The /etc/fstab configuration files for the clients have the line
rg.local:/home/pi /mnt/g nfs defaults 0 0
I can get 15 of the clients to connect to the server, but trying to connect a 16th machine hangs. I have checked that it does not matter which 15 connect.

I have increased the number of initial threads to 20 in the file /etcdefault/nfs-kernel-server but this has been of no help.

I have been trying to figure out why this is happening and I am stumped.

Does anyone understand what is causing this? Is there some limit set by SystemD, to manage resources?

After some research I found this issue occurs only in NFSv4.

User avatar
DougieLawson
Posts: 34166
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website

Re: NFS server limiting the number of connections. Why?

Sun Jun 24, 2018 8:53 am

What value do you have in /etc/default/nfs-kernel-server for

Code: Select all

# Number of servers to start up
RPCNFSDCOUNT=8
Microprocessor, Raspberry Pi & Arduino Hacker
Mainframe database troubleshooter
MQTT Evangelist
Twitter: @DougieLawson

2012-18: 1B*5, 2B*2, B+, A+, Z, ZW, 3Bs*3, 3B+

Any DMs sent on Twitter will be answered next month.

tgwena
Posts: 11
Joined: Tue Nov 06, 2012 12:30 am

Re: NFS server limiting the number of connections. Why?

Sun Jun 24, 2018 1:15 pm

DougieLawson wrote:
Sun Jun 24, 2018 8:53 am
What value do you have in /etc/default/nfs-kernel-server for

Code: Select all

# Number of servers to start up
RPCNFSDCOUNT=8
I put it up to 20.

MarkJ62
Posts: 24
Joined: Mon Dec 17, 2012 11:55 am
Location: Sydney, Australia
Contact: Website

Re: NFS server limiting the number of connections. Why?

Sun Sep 09, 2018 7:16 am

I’m having a similar issue. I have increased the limit to 20 as suggested by Dougie Lawson. I have 18 nodes that are listed by exportfs but I can’t seem to get past 15 connected.

If I pick one that hasn’t connected and manually issue the mount command it just sits there. Doing a ”netstat -a | grep nfs” on the nfs server shows it as having established a connection and some random port assignment, but the nfs client seems to hang. I’m running Stretch and the 4.14 kernel on all the nodes and nfs server.

Jag_ESU
Posts: 4
Joined: Wed Oct 24, 2018 10:50 pm

Re: NFS server limiting the number of connections. Why?

Wed Oct 24, 2018 11:01 pm

I am doing a cluster with 16 Raspberry Pi's, After 15 Pi's connected well in NFS,
the 16th one is not connecting in NFS. The Client is hanging for indefinite period while doing the mount.
I saw the thread about you guys.

Did you guys found any solution for it ?

ejolson
Posts: 2160
Joined: Tue Mar 18, 2014 11:47 am

Re: NFS server limiting the number of connections. Why?

Fri Oct 26, 2018 4:09 am

Jag_ESU wrote:
Wed Oct 24, 2018 11:01 pm
I am doing a cluster with 16 Raspberry Pi's, After 15 Pi's connected well in NFS,
the 16th one is not connecting in NFS. The Client is hanging for indefinite period while doing the mount.
I saw the thread about you guys.

Did you guys found any solution for it ?
Welcome to the forum! It is a strange limitation which apparently only affects NFS version 4 mounts. Unfortunately, I don't have the hardware to test this myself. If you explicitly specify vers=3 in the mount command, do things work as expected with more clients?

Jag_ESU
Posts: 4
Joined: Wed Oct 24, 2018 10:50 pm

Re: NFS server limiting the number of connections. Why?

Tue Oct 30, 2018 3:37 pm

Did you guys find any solution as i have having 16 Pi's & struggling to connect the last one. Please let me know if any success is there.

Jag_ESU
Posts: 4
Joined: Wed Oct 24, 2018 10:50 pm

Re: NFS server limiting the number of connections. Why?

Tue Oct 30, 2018 3:51 pm

Nope, it's won't work. I have already tried that,
Then i tried to make changes in default kernel folder by removing version 4 it didn't work
I tried to increase the thread count from 8 to 16 in default kernel folder it didn't work also.
I got an article that for linux default tcp_slot_table_entries=16, need to increase it to 128 .
But i couldn't find the file in Rasp Pi. So made a file inside etc/modprobe.d as sunrpc.conf
but still no luck

Jag_ESU
Posts: 4
Joined: Wed Oct 24, 2018 10:50 pm

Re: NFS server limiting the number of connections. Why?

Tue Oct 30, 2018 6:55 pm

Finally, I made it guys...

1. Change the sudo nano /etc/default/nfs-kernel-server

RPCNFSDCOUNT = 16
RPCMOUNTDOPTS= “ --manage-gids --no-nfs-version 3“

2. Create sunrpc.conf file in /etc/modprobe.d
Add below entries in the sunrpc.conf

options sunrpc tcp_slot_table_entries=128
options sunrpc tcp_max_slot_table_entries=128

After that,
Reboot the Raspberry pi then,

Restart NFS Server
sudo /etc/init.d/nfs-kernel-server restart

Restart the client as well.

That's all guys...

Return to “Networking and servers”