On a side note, this option is set to disabled in clustered Data ONTAP by default.
-----Original Message----- From: Parisi, Justin Sent: Monday, August 12, 2013 12:21 PM To: 'Peter D. Gray'; toasters@teaparty.net Subject: RE: options weirdness
The reason your systems were allowing mounts was likely due to them using NFSv4.
However, it's important to note there is a distinction between mount rootonly and NFS rootonly.
Mount rootonly has been around for years, pre-dating 8.x ONTAP. NFS rootonly is fairly new, but was definitely around in at least 8.1.2:
fas3170-rtp> version NetApp Release 8.1.2 7-Mode: Tue Oct 30 19:56:51 PDT 2012 fas3170-rtp*> options nfs.nfs nfs.nfs_rootonly off
NFS rootonly was not in 7.3.6, which was the version I checked, so likely came about between 7.3.6 and 8.1.2.
Mount rootonly means mounts can come only from privileged ports, which are port numbers lower than 1024. This is from the client side. The filer always uses port 4046 for mounts, regardless of what the client does:
[root@centos64 ~]# rpcinfo -p 10.61.72.35 | grep mount 100005 3 tcp 4046 mountd 100005 2 tcp 4046 mountd 100005 1 tcp 4046 mountd 100005 3 udp 4046 mountd 100005 2 udp 4046 mountd 100005 1 udp 4046 mountd
This is an example of a mount where rootonly is set to on, which is the default behavior.
110 4.145080 10.61.179.150 10.61.72.35 MOUNT 162 V3 MNT Call (Reply In 111) /vol/unix User Datagram Protocol, Src Port: 898 (898), Dst Port: acp-proto (4046)
111 4.145816 10.61.72.35 10.61.179.150 MOUNT 118 V3 MNT Reply (Call In 110) User Datagram Protocol, Src Port: acp-proto (4046), Dst Port: 898 (898) Accept State: RPC executed successfully (0)
The mount source port from the client is 898, which is lower than 1024. Thus the mount succeeds.
When the source port is higher than 1024 for mounts and rootonly is on, the mount fails. This can be set via fstab using noresvport:
[root@centos64 /]# cat /etc/fstab | grep 7mode 10.61.72.35:/vol/unix /7mode nfs nfsvers=3,intr,noauto,user,noresvport 0 0
121 4.933607 10.61.179.150 10.61.72.35 MOUNT 162 V3 MNT Call (Reply In 122) /vol/unix User Datagram Protocol, Src Port: 48317 (48317), Dst Port: acp-proto (4046)
122 4.933848 10.61.72.35 10.61.179.150 MOUNT 62 V3 MNT Reply (Call In 121) User Datagram Protocol, Src Port: acp-proto (4046), Dst Port: 48317 (48317) Reject State: AUTH_ERROR (1) Auth State: rejected for security reasons (5)
sh-4.1$ mount 7mode mount.nfs: access denied by server while mounting 10.61.72.35:/vol/unix
An error is logged to the filer console:
Mon Aug 12 09:36:17 EST [MNTPool09:warning]: Client 10.61.179.150 (xid 2871886776), from mount is trying to mount from a nonreserved port = 48317 as uid = 0
When mount rootonly is set to off, mounts to non-reserved ports succeed:
73 1.849448 10.61.179.150 10.61.72.35 MOUNT 154 V3 MNT Call (Reply In 74) /vol/unix User Datagram Protocol, Src Port: 60365 (60365), Dst Port: acp-proto (4046)
74 1.849913 10.61.72.35 10.61.179.150 MOUNT 118 V3 MNT Reply (Call In 73) Internet Protocol Version 4, Src: 10.61.72.35 (10.61.72.35), Dst: 10.61.179.150 (10.61.179.150) Accept State: RPC executed successfully (0)
When mount rootonly is set to off but nfs rootonly is set to on and a mount is attempted to a non-privileged port, the mount returns successful but once the NFS portion starts the mount bombs out:
72 1.963629 10.61.179.150 10.61.72.35 MOUNT 162 V3 MNT Call (Reply In 73) /vol/unix User Datagram Protocol, Src Port: 37285 (37285), Dst Port: acp-proto (4046)
73 1.963926 10.61.72.35 10.61.179.150 MOUNT 118 V3 MNT Reply (Call In 72) User Datagram Protocol, Src Port: acp-proto (4046), Dst Port: 37285 (37285) Accept State: RPC executed successfully (0)
92 1.966387 10.61.179.150 10.61.72.35 NFS 210 V3 FSINFO Call (Reply In 93), FH:0x7a8971d1 Transmission Control Protocol, Src Port: 44569 (44569), Dst Port: nfs (2049), Seq: 89, Ack: 57, Len: 144
93 1.966837 10.61.72.35 10.61.179.150 NFS 90 V3 FSINFO Reply (Call In 92) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 44569 (44569), Seq: 57, Ack: 233, Len: 24 Reject State: AUTH_ERROR (1) Auth State: rejected for security reasons (5)
An error will show up on the filer console:
Mon Aug 12 09:40:30 EST [fas3170-rtp:nfsd.rpc.request.bad:warning]: Client 10.61.179.150 is sending bad RPC requests with error: RPC version mismatch or authentication error (73). Mon Aug 12 09:40:30 EST [fas3170-rtp:nfsd.auth.status.bad:warning]: Client 10.61.179.150 has an authentication error 5. Mon Aug 12 09:40:30 EST [fas3170-rtp:nfsd.rpc.request.bad:warning]: Client 10.61.179.150 is sending bad RPC requests with error: RPC version mismatch or authentication error (73). Mon Aug 12 09:40:30 EST [fas3170-rtp:nfsd.auth.status.bad:warning]: Client 10.61.179.150 has an authentication error 5.
If you were to try to mount and use a port other than 2049 with nfs.nfs_rootonly off, the mount would fail even as root, as the filer only allows NFS to port 2049:
[root@centos64 /]# rpcinfo -p 10.61.72.35 | grep nfs 100003 4 tcp 2049 nfs 100003 3 tcp 2049 nfs 100003 2 tcp 2049 nfs 100003 3 udp 2049 nfs 100003 2 udp 2049 nfs
[root@centos64 /]# cat /etc/fstab | grep 7mode 10.61.72.35:/vol/unix /7mode nfs nfsvers=3,intr,noauto,user,noresvport,port=2040 0 0
[root@centos64 /]# mount 7mode mount.nfs: requested NFS version or transport protocol is not supported
That's because portmapper asks the filer which port it allows NFS on and the filer replies "2049" and the client disallows the request:
61 2.624721 10.61.179.150 10.61.72.35 Portmap 98 V2 GETPORT Call (Reply In 65) NFS(100003) V:3 UDP 65 2.624969 10.61.72.35 10.61.179.150 Portmap 70 V2 GETPORT Reply (Call In 61) Port:2049
So again, the client controls which port NFS comes in on, but the filer only allows one port for NFS and one port for MOUNT calls.
Toggling the options is intended for the following:
- allowing more than 1024 mount points on a client - special use cases where clients don't use privileged mount or NFS ports
The security hole wouldn't be on the filer in this case, other than what it allows. And NFS wouldn't be possible without mounting first in v3, so if rootonly is set to on for mount, then the NFS option isn't really applicable since the client wouldn't be able to get that far.
If NFSv4, then there is no mount protocol - only port 2049. So the rootonly mount option wouldn't apply in that case, as it only looks for MOUNT calls. In that case, NFS rootonly would matter.
In the case of NFSv4, you would want to enable nfs rootonly to prevent non-privileged ports to access.
This is NFSv4 with mountrootonly off and nfsrootonly on:
[root@centos64 /]# mount -t nfs4 -o noresvport 10.61.72.35:/vol/unix /7mode mount.nfs4: access denied by server while mounting 10.61.72.35:/vol/unix
110 2.985435 10.61.179.150 10.61.72.35 NFS 202 V4 Call (Reply In 111) PUTROOTFH | GETATTR Transmission Control Protocol, Src Port: 58233 (58233), Dst Port: nfs (2049), Seq: 45, Ack: 29, Len: 136
111 2.986164 10.61.72.35 10.61.179.150 NFS 90 V4 Reply (Call In 110) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 58233 (58233), Seq: 29, Ack: 181, Len: 24 Reject State: AUTH_ERROR (1)
This is NFSv4 with mountrootonly on and nfsrootonly off:
[root@centos64 /]# mount -t nfs4 -o noresvport 10.61.72.35:/vol/unix /7mode [root@centos64 /]#
45 1.577658 10.61.179.150 10.61.72.35 NFS 202 V4 Call (Reply In 46) PUTROOTFH | GETATTR Transmission Control Protocol, Src Port: 48066 (48066), Dst Port: nfs (2049), Seq: 45, Ack: 29, Len: 136
46 1.578116 10.61.72.35 10.61.179.150 NFS 326 V4 Reply (Call In 45) PUTROOTFH | GETATTR Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 48066 (48066), Seq: 29, Ack: 181, Len: 260 Accept State: RPC executed successfully (0)
Note that mountrootonly didn't help here as the traffic all uses NFS ports. But again, we always use port 2049 for NFS on the filer.
I agree that the default behavior of the filer should be to set nfsrootonly to ON when NFSv4 is enabled. I will raise a bug against that....
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Peter D. Gray Sent: Sunday, August 11, 2013 7:26 PM To: toasters@teaparty.net Subject: options weirdness
Can someone who knows tell me whats going on here?
We recently discovered a security issue with our netapp filers when using NFS. The netapps where allowing NFS mounts and operations from non-privilaged client ports.
Investigation found the options nfs.nfs_rootonly and nfs.mount_rootonly options.
nfs.mount_rootonly was true, but nfs.nfs_rootonly was false.
I was kind of surprized by this, since I had no recollection of ever altering either of these options yet the settings were identical across all our filers. The environment is:
3170A running 8.2 3240 running 8.2 3240 running 8.1 (reverted from 8.2)
Ok, so I asked another nearby site to check there filers for me and they tell me their filers have no such setting, all on 8.1
So, I guess my questions are:
1) has ONTAP always allowed NFS from non-privilaged ports?
2) was nfs.nfs_rootonly introduced in 8.2 and why is the default off?
3) why does this setting stay around after revert to 8.1?
It would seem to me that allowing NFS from non-privilages ports is kind of bad.
Any help appreciated.
Regards, pdg
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters