It is very minor. I would VERIFY that the switch ports though are set properly:
"spanning-tree port type edge trunk"
--> I presume you are using VLAN tags for ESXi/NFS.
This allows the ports to come up quickly rather than waiting for STP to do its thing (45 seconds or so)
You would need to modify the switch first. Most NX-OS code already has jumbo frames enabled. You should verify just in case.
Then you will need to set the base interface (ifgrp/a0a) to MTU of 9000. If the port is in a broadcast domain, but not in use by anything, I would remove from the broadcast domain.
Then set the port to mtu of 9000 (net port modify -node xxx -port a0a -mtu 9000) and wait. When it comes back, wait for the ifgrp to return to "full" (ifgrp show)
After your ifgrps are set, give a go a and change ONTAP (broadcast-domain modify -broadcast-domain esx -mtu 9000) which will change all ports at the same time
After the ONTAP side is set: VERIFY you at least have jumbo frames working across the switch:
net ping -lif esx-nfs-01 -vserver esxi-svm -destination xx.xx.xx.xx (where that is the IP of the esx-nfs-02 LIF)
net ping -lif esx-nfs-01 -vserver esxi-svm -disable true -packet 5000 -destination xx.xx.xx.xx (where that is the IP of the esx-nfs-02 LIF)
-> this will test a 5000 mtu packet between NetApp nodes.
If that works continue on
Modify the vswitch on ESX to be 9000 then go to each VMK and set the MTU to 9000 also.
From the NetApp you should be able to do that last ping and test jumbo frames.
If all is well...great. If not, revert back to 1500 and try again. NFS is slightly forgiving.
--tmac
Tim McCarthy, Principal Consultant