What is the MTU size on your WAN link?
----- "Carl Howell" chowell@uwf.edu wrote:
What would be the optimal NFS settings for a filer on a 1Gb network with a 14ms rtt? We have moved our R200 to a remote DC and have seen NFS performance plummet. The NFS traffic is all backups.
Thanks,
-Carl
9000. And to be clear, these are backups that are going to the R200.
--Carl
-----Original Message----- From: Tom Samplonius [mailto:tom@samplonius.org] Sent: Wednesday, June 06, 2007 7:53 PM To: Carl Howell Cc: toasters@mathworks.com Subject: Re: NFS backups over 1Gb WAN
What is the MTU size on your WAN link?
----- "Carl Howell" chowell@uwf.edu wrote:
What would be the optimal NFS settings for a filer on a 1Gb network with a 14ms rtt? We have moved our R200 to a remote DC and have seen NFS performance plummet. The NFS traffic is all backups.
Thanks,
-Carl
Are you using NFS over the WAN? If so, what are your mount options?
If not, then are you using SnapVaul/SnapMirror over the WAN or something else?
On 6/7/07, Carl Howell chowell@uwf.edu wrote:
- And to be clear, these are backups that are going to the R200.
--Carl
-----Original Message----- From: Tom Samplonius [mailto:tom@samplonius.org] Sent: Wednesday, June 06, 2007 7:53 PM To: Carl Howell Cc: toasters@mathworks.com Subject: Re: NFS backups over 1Gb WAN
What is the MTU size on your WAN link?
----- "Carl Howell" chowell@uwf.edu wrote:
What would be the optimal NFS settings for a filer on a 1Gb network with a 14ms rtt? We have moved our R200 to a remote DC and have seen NFS performance plummet. The NFS traffic is all backups.
Thanks,
-Carl
Yes. NFS. Mount options: rw,bg,hard,intr,vers=3,proto=tcp,rsize=32768,wsize=32768.
I'm not sure you would call it a WAN. I believe the correct acronym is LFN or Long Fat Network(right?). We have VLAN's that we ride over the Florida LamdaRail to our colo. The RTT is around 13-14ms.
I know there are appliances out there that can solve this problem, but these backups won't be done this way for much longer, and I just wanted to see if I could get the math right.
For our filers, I did adjust the option snapmirror.window_size to 1750000 (1Gb * .014/8), and that is working fine.
--Carl
From: tmac [mailto:tmacmd@gmail.com] Sent: Thursday, June 07, 2007 8:29 AM To: Carl Howell Cc: toasters@mathworks.com Subject: Re: NFS backups over 1Gb WAN
Are you using NFS over the WAN? If so, what are your mount options?
If not, then are you using SnapVaul/SnapMirror over the WAN or something else?
On 6/7/07, Carl Howell chowell@uwf.edu wrote:
9000. And to be clear, these are backups that are going to the R200.
--Carl
-----Original Message----- From: Tom Samplonius [mailto:tom@samplonius.org] Sent: Wednesday, June 06, 2007 7:53 PM To: Carl Howell Cc: toasters@mathworks.com Subject: Re: NFS backups over 1Gb WAN
What is the MTU size on your WAN link?
----- "Carl Howell" <chowell@uwf.edu > wrote:
What would be the optimal NFS settings for a filer on a 1Gb network with a 14ms rtt? We have moved our R200 to a remote DC and have seen NFS performance plummet. The NFS traffic is all backups.
Thanks,
-Carl
You may way to try UDP instead of TCP. and/or reduce the rsize/wsize to 8k or 1k to see if it makes a difference. If it does, then keep increasing it (1K, 2K, 4K, 8K, 16K) until it is no longer useful then drop it back one. The retrans and timeo options may also be useful as well.
Also, if you stay with TCP, you may want to investigate changing the TCP window size on the NFS client.
On 6/7/07, Carl Howell chowell@uwf.edu wrote:
Yes. NFS. Mount options: rw,bg,hard,intr,vers=3,proto=tcp,rsize=32768,wsize=32768.
I'm not sure you would call it a WAN. I believe the correct acronym is LFN or Long Fat Network(right?). We have VLAN's that we ride over the Florida LamdaRail to our colo. The RTT is around 13-14ms.
I know there are appliances out there that can solve this problem, but these backups won't be done this way for much longer, and I just wanted to see if I could get the math right.
For our filers, I did adjust the option snapmirror.window_size to 1750000 (1Gb * .014/8), and that is working fine.
--Carl
*From:* tmac [mailto:tmacmd@gmail.com] *Sent:* Thursday, June 07, 2007 8:29 AM *To:* Carl Howell *Cc:* toasters@mathworks.com *Subject:* Re: NFS backups over 1Gb WAN
Are you using NFS over the WAN? If so, what are your mount options?
If not, then are you using SnapVaul/SnapMirror over the WAN or something else?
On 6/7/07, *Carl Howell* chowell@uwf.edu wrote:
- And to be clear, these are backups that are going to the R200.
--Carl
-----Original Message----- From: Tom Samplonius [mailto:tom@samplonius.org] Sent: Wednesday, June 06, 2007 7:53 PM To: Carl Howell Cc: toasters@mathworks.com Subject: Re: NFS backups over 1Gb WAN
What is the MTU size on your WAN link?
----- "Carl Howell" <chowell@uwf.edu > wrote:
What would be the optimal NFS settings for a filer on a 1Gb network with a 14ms rtt? We have moved our R200 to a remote DC and have seen NFS performance plummet. The NFS traffic is all backups.
Thanks,
-Carl
-- --tmac
RedHat Certified Engineer
Interesting topic and great suggestions.
I'm guessing your network topology is something like:
Local Ethernet LAN <-> WAN <-> Remote Ethernet LAN
Carl if I was you I'd also try jumbo Ethernet frames and see if that makes a difference, especially since you mentioned the MTU on your WAN is 9000 bytes. Frankly I've had mixed results with jumbo frames, but in your case it may make a difference given how the WAN MTU matches up.
A quick and dirty test would be to pick a client in your local LAN. Copy over a large file over NFS to your R200, check stats. Change to jumbo frames, reboot the client (so you clear the cache) and copy over the same file again, and check stats.
Are you CPU-bound on your R200? Otherwise, can you check the health of your routers/switches along the network path and see if there's a bottleneck there somewhere?
You didn't mention your clients or backup application, they can make a big difference (eg: Linux 2.6 kernel
2.4 kernel)
Regards, Sandeep Cariapa Sales Engineer IBRIX
--- tmac tmacmd@gmail.com wrote:
You may way to try UDP instead of TCP. and/or reduce the rsize/wsize to 8k or 1k to see if it makes a difference. If it does, then keep increasing it (1K, 2K, 4K, 8K, 16K) until it is no longer useful then drop it back one. The retrans and timeo options may also be useful as well.
Also, if you stay with TCP, you may want to investigate changing the TCP window size on the NFS client.
On 6/7/07, Carl Howell chowell@uwf.edu wrote:
Yes. NFS. Mount options:
rw,bg,hard,intr,vers=3,proto=tcp,rsize=32768,wsize=32768.
I'm not sure you would call it a WAN. I believe
the correct acronym is LFN
or Long Fat Network(right?). We have VLAN's that
we ride over the Florida
LamdaRail to our colo. The RTT is around 13-14ms.
I know there are appliances out there that can
solve this problem, but
these backups won't be done this way for much
longer, and I just wanted to
see if I could get the math right.
For our filers, I did adjust the option
snapmirror.window_size to 1750000
(1Gb * .014/8), and that is working fine.
--Carl
*From:* tmac [mailto:tmacmd@gmail.com] *Sent:* Thursday, June 07, 2007 8:29 AM *To:* Carl Howell *Cc:* toasters@mathworks.com *Subject:* Re: NFS backups over 1Gb WAN
Are you using NFS over the WAN? If so, what are your mount options?
If not, then are you using SnapVaul/SnapMirror
over the WAN or something
else?
On 6/7/07, *Carl Howell* chowell@uwf.edu wrote:
- And to be clear, these are backups that are
going to the R200.
--Carl
-----Original Message----- From: Tom Samplonius [mailto:tom@samplonius.org] Sent: Wednesday, June 06, 2007 7:53 PM To: Carl Howell Cc: toasters@mathworks.com Subject: Re: NFS backups over 1Gb WAN
What is the MTU size on your WAN link?
----- "Carl Howell" <chowell@uwf.edu > wrote:
What would be the optimal NFS settings for a
filer on a 1Gb network
with a 14ms rtt? We have moved our R200 to a
remote DC and have seen
NFS performance plummet. The NFS traffic is all
backups.
Thanks,
-Carl
-- --tmac
RedHat Certified Engineer
-- --tmac
RedHat Certified Engineer
Sandeep Cariapa
-
Carl Howell -
Sandeep Cariapa -
tmac -
Tom Samplonius