Ubuntu – Unable to NFS mount from Ubuntu 14.04.1 LTS against QNAP NAS

14.04mountnetworkingnfstimeout

Note, this is a cross posting question in the QNAP NAS community here: http://forum.qnap.com/viewtopic.php?f=35&t=96526&p=427018#p427018

Any comments and suggestions, as well as pointers to relevant information pieces are very appreciated.

I cannot NFS mount from my NFS client running Ubuntu 14.04.1 (LTS) against my NFS server (QNAP NAS). My environment is:

NFS Server: QNAP TS-669 Pro running firmware 4.1.0 (dated: 2014/06/12)
NFS Client: ECS LIVA (a small barebone PC) running Ubuntu 14.04.1 LTS Desktop.
The two systems are connected via 1000Base-T Ethernet and IP reacheable.
Name resolution is done by local registry (/etc/hosts) and getent hosts host-name command returns the correct and consistent IP address on both nodes.
NFS service is enabled on the NFS server and NO_LIMIT access right is given on the specific share folder, namely "/nfs", on the "NFS host access" tab of the "Shared Folders" configuration app: in fact, I can confirm it is world NFS exported via issueing the "exportfs -rva" command on NAS.
Because Ubuntu (the NFS client) does not install NFS client packages by default, I explicitly installed the nfs-common package as described in here: Setting Up NFS HOW-TO ( https://help.ubuntu.com/community/SettingUpNFSHowTo#NFS_Client ); The rpcbind package seems to be installed by default.

On the NFS client, If I run the command "mount -t nfs nas:/nfs /mnt", it gives the output "mount.nfs: Connection timed out" after five or 10 minutes later. The same result is returned even if I specify NFS version 3 protocol with -overs=3 option while trying to NFS mount.
Also the command "showmount -e" lists up the exported NFS shared folder (directory) names eventually, but it also takes five or 10minutes to complete.

On the NFS server, the "exportfs -rva" command returns the following warning message, but I do believe the message does not relate with the problem (I am accessing the NAS via SSH in this code exmple):

[~] # exportfs -rva
exportfs: /etc/exports [1]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/share/MD0_DATA/Public".
Assuming default behaviour ('no_subtree_check').   NOTE: this default has changed since nfs-utils version 1.0.x

exportfs: /etc/exports [2]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/share/MD0_DATA/nfs".
Assuming default behaviour ('no_subtree_check').   NOTE: this default has changed since nfs-utils version 1.0.x

exporting *:/share/MD0_DATA/nfs exporting *:/share/MD0_DATA/Public

On the NFS client, the mount command takes long time (more than five minutes) to complete. I specified the vers=3 option, because I understand QNAP does not support NFS V4 by default and NFS V3 suffices my requirement. It does not matter whether or not specifying the tcp and/or nolock options (same behavior).

root@livak5:~# mount -t nfs -vvv -overs=3,tcp,nolock nas:/share/MD0_DATA/nfs /mnt
mount: fstab path: "/etc/fstab"
mount: mtab path:  "/etc/mtab"
mount: lock path:  "/etc/mtab~"
mount: temp path:  "/etc/mtab.tmp"
mount: UID:        0
mount: eUID:       0
mount: spec:  "nas:/share/MD0_DATA/nfs"
mount: node:  "/mnt"
mount: types: "nfs"
mount: opts:  "vers=3,tcp,nolock"
mount: external mount: argv[0] = "/sbin/mount.nfs"
mount: external mount: argv[1] = "nas:/share/MD0_DATA/nfs"
mount: external mount: argv[2] = "/mnt"
mount: external mount: argv[3] = "-v"
mount: external mount: argv[4] = "-o"
mount: external mount: argv[5] = "rw,vers=3,tcp,nolock"
mount.nfs: timeout set for Sun Aug 24 11:24:44 2014
mount.nfs: trying text-based options 'vers=3,tcp,nolock,addr=192.168.11.50'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 192.168.11.50 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: trying 192.168.11.50 prog 100005 vers 3 prot TCP port 41687
mount.nfs: mount(2): Connection timed out
mount.nfs: trying text-based options 'vers=3,tcp,nolock,addr=192.168.11.50'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 192.168.11.50 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: trying 192.168.11.50 prog 100005 vers 3 prot TCP port 41687
mount.nfs: mount(2): Connection timed out
mount.nfs: trying text-based options 'vers=3,tcp,nolock,addr=192.168.11.50'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 192.168.11.50 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: trying 192.168.11.50 prog 100005 vers 3 prot TCP port 41687
mount.nfs: mount(2): Connection timed out
mount.nfs: Connection timed out

On the NFS client, portmapper seeems working just fine against NFS version 2, 3, and 4:

root@livak5:~# rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  57148  status
    100024    1   tcp  52831  status

On the NFS server, I tried to check if the pormapper is runing on it from the NFS client, since it does not have the rpcinfo command installed:

root@livak5:~# nc -zv nas 111
Connection to nas 111 port [tcp/sunrpc] succeeded!
root@livak5:~# rpcinfo -s nas
   program version(s) netid(s)                         service     owner
    100000  2,3,4     local,udp,tcp                    portmapper  superuser
    100011  2,1       tcp,udp                          rquotad     superuser
    100005  3,2,1     tcp,udp                          mountd      superuser
    100003  3,2       udp,tcp                          nfs         superuser
    100227  3,2       udp,tcp                          -           superuser
    100021  4,3,1     tcp,udp                          nlockmgr    superuser
    100024  1         tcp,udp                          status      superuser

The later command (rpcinfo) takes long time (more than five minutes) to complete which replicates the problem root cause, I believe.
Please note that on both the TCP ports, 2049 and 41687, appropriate daemon processes are listening on NAS. I can confirm this fact since the nc command returns instantly on the NFS client against NAS as shown in the following output:

root@livak5:~# nc -zv nas 2049
Connection to nas 2049 port [tcp/nfs] succeeded!
root@livak5:~# nc -zv nas 41687
Connection to nas 41687 port [tcp/*] succeeded!

Strangely enough, I can NFS version 3 mount on NAS itself as shown in the following output (I am accessing the NAS via SSH in this code exmple):

[~] # mkdir /mnt2
[~] # mount -overs=3 nas:/share/MD0_DATA/public /mnt2
[~] # df -k
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/ram0               154691    137854     16837  89% /
devtmpfs               1531580         4   1531576   0% /dev
tmpfs                    65536       160     65376   0% /tmp
/dev/md9                521684    126312    395372  24% /mnt/HDA_ROOT
/dev/md0             11622485880 410664920 11211296672   4% /share/MD0_DATA
/dev/md13               379888    259868    120020  68% /mnt/ext
tmpfs                    32768         0     32768   0% /.eaccelerator.tmp
nas:/share/MD0_DATA/public/11622485888 411189216 11211296672   4% /mnt2

Although it looks as if I have some sort of blocked ports problem on NFS Client, but it seems Ubuntu 14.04.1 does not enable ufw (uncomplicated firewall, it is actually a front-end to iptables) by default as shown in the following wiki document: Uncomplicated Firewall (somehow, I cannot put the wiki URL in here).
I can confirm nothing blocked on it by running command on the NFS client:

root@livak5:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Best Answer

It turns out the problem root cause is the firmware bug of Ethernet Switch I am using (NetGEAR GS116Ev2). After updating the firmware to 2.0.1.17 from 2.0.0.23, the problem is gone.

Related Solutions

Ubuntu – Diskless with Ubuntu 12.04

Your first attempt seems to be wrong because the full path is not specified.

Your second attempt seems to be that there is a mismatch in nfs versions: The initramfs loads version 4 but the server only serves version 3.

Probably this might help:

https://help.ubuntu.com/community/SettingUpNFSHowTo

You might try to compile the TFTP-Served kernel with nfs-3-support, see lsinitramfs initrd.img-3.5.0-25-generic to check what's inside.

Ubuntu – Firewall problem using autofs with NFS-exported mounts

In the end itwas not that complicated, following the hints in Securing NFS. I modified the files /etc/default/nfs-common and /etc/default/nfs-kernel-server according:

nfs-common:

.
.
# Options for rpc.statd.
#   Should rpc.statd listen on a specific port? This is especially useful
#   when you have a port-based firewall. To use a fixed port, set this
#   this variable to a statd argument like: "--port 4000 --outgoing-port 4001".
#   For more information, see rpc.statd(8) or http://wiki.debian.org/SecuringNFS
STATDOPTS="--port 32765 --outgoing-port 32766"
.
.

nfs-kernel-server:

.
.
# Options for rpc.mountd.
# If you have a port-based firewall, you might want to set up
# a fixed port here using the --port option. For more information,
# see rpc.mountd(8) or http://wiki.debian.org/SecuringNFS
# To disable NFSv4 on the server, specify '--no-nfs-version 4' here
RPCMOUNTDOPTS="--manage-gids --port 32767"
.
.

Why those ports? As 32767 is the highest 15bit-number, it is very unlikely that these ports are already in use by something else.

I am not using quotas, so I did not modify /etc/default/quota as suggested. And I had to reboot after I made these changes. Then I saw the result with

$ sudo netstat -nalp | grep rpc

tcp        0      0 0.0.0.0:32767           0.0.0.0:*               LISTEN      1018/rpc.mountd 
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      735/rpcbind     
tcp        0      0 0.0.0.0:32765           0.0.0.0:*               LISTEN      806/rpc.statd   
tcp6       0      0 :::32767                :::*                    LISTEN      1018/rpc.mountd 
tcp6       0      0 :::111                  :::*                    LISTEN      735/rpcbind     
tcp6       0      0 :::32765                :::*                    LISTEN      806/rpc.statd   
udp        0      0 0.0.0.0:875             0.0.0.0:*                           735/rpcbind     
udp        0      0 127.0.0.1:982           0.0.0.0:*                           806/rpc.statd   
udp        0      0 0.0.0.0:32765           0.0.0.0:*                           806/rpc.statd   
udp        0      0 0.0.0.0:32767           0.0.0.0:*                           1018/rpc.mountd 
udp        0      0 0.0.0.0:111             0.0.0.0:*                           735/rpcbind     
udp6       0      0 :::875                  :::*                                735/rpcbind     
udp6       0      0 :::32765                :::*                                806/rpc.statd   
udp6       0      0 :::32767                :::*                                1018/rpc.mountd 
udp6       0      0 :::111                  :::*                                735/rpcbind     
unix  2      [ ACC ]     STREAM     LISTENING     11412    735/rpcbind         /run/rpcbind.sock
unix  2      [ ]         DGRAM                    9521     806/rpc.statd       
unix  2      [ ]         DGRAM                    9614     1018/rpc.mountd     
unix  3      [ ]         STREAM     CONNECTED     11721    862/rpc.idmapd      
unix  3      [ ]         STREAM     CONNECTED     11722    862/rpc.idmapd

As you can see, the ports rpc.mountd and rpc.statd are listening to are now static.

When entering showmounton the client(here 192.168.192.20), Wireshark shows the communication (server is 192.168.192.111). Important here: The GETPORT Call and the GETPORT reply, which returns Port:32767. The communication then uses this port.

Now you should be able to modify the firewall rules according, and then use showmountand autofs through firewall.

Just for the record

Following the hints in the comments and my own experience, I found different behaviour in different distributions:

In current raspbian jessie (based on debian), there is a service nfs-common (file /etc/init.d/nfs-common), which when enabled starts e.g. rpc.statd at boot, respecting the port settings in /etc/default/nfs-common.
In current Ubuntu 16.04 there is no such service. rpc.statdis not started at boot, as it is not needed with NFS V4. But as soon as mount .... -o nfsvers=3 is done, rpc.statd is started, respecting the port settings in /etc/default/nfs-common.

I did not find a consistent documentation on that; in How to configure NFS the file /etc/init.d/nfs-common is explicitly mentioned, although it is not in the package. If anyone has hints/links on that it would be richly deserved.

One more remark: man rpc.mountd and man rpc.statd say (for option --port):

"If this option is not specified, rpc.statd will try to consult /etc/services, if gets port succeed, set the same port for all listener socket, otherwise chooses a random ephemeral port for each listener socket."

Even when setting the ports in /etc/services (as suggested in the above mentioned wiki), this did not work. So modifying the files in /etc/default seems mandatory - the man pages are not correct at that point.

Best Answer

Related Solutions

Ubuntu – Diskless with Ubuntu 12.04

Ubuntu – Firewall problem using autofs with NFS-exported mounts

Related Question