Hi Jack,
I am using the same win2k server to copy to/from the NetApp and to/from another Win2k server. The only difference is which interface is used (the Win2k server has 100mbit and GigE to different vlans). I am actually transferring a 30 gigabyte file, not 30 megabyte. Your suggestion to defragment the Win2k box was valid, as the test server was slightly fragmented. I did a quick analysis after the defrag and this is what I found:
Win2k server writing 30GB file over a 100mbit link to a Win2k Server: 12.2MB/sec (link saturated) Win2k server reading 30GB file over a 100mbit link from a Win2k Server: 12.13MB/sec (link saturated) Win2k server writing 30GB file over a GigE link to a NetApp 760: 7MB/sec Win2k server reading 30GB file over a GigE link from a NetApp 760: 25MB/sec
note: these are the best readings after playing with several Win2k gigE settings (including your suggestion of TCP offload). I guess my question is, should I expect better performance from the NetApp?
-----Original Message----- From: Jack Lyons [mailto:jack.lyons@martinagency.com] Sent: Friday, April 02, 2004 9:43 AM To: 'Tim Driggers'; 'NetApp List (E-mail)' Subject: RE: CIFS performance Issue
There are a lot of variables involved.
What client are you using for copying the file to and from the NetApp. Are you using the Win2k server you are comparing it to?
I think some of what is going is that the Win2k maybe caching the writes to the file, but when writing to the Filer it is not caching the reads (or if it is - it isn't doing it efficiently).
I bet if you looked at the defrag map of the Win2k box it would be moderately fragmented.
Also, for a Gig link, a 30 MB file is not that big. Try using a 300 MB file.
We looked at several different solutions using different protocols and CIFS clients. Here is a snapshot of my results.
Single Client WinXP client writing 300MB file over a GigE link to a Win2k Server: 40MB/s WinXP client reading 300MB file over a GigE link from a Win2k Server: 26MB/s MacOSX running the Dave Client writing a 300MB file over a GigE link to a FAS960: 22MB/s MacOSX running the Dave Client reading a 300MB file over a GigE link from a FAS960: 38MB/s
However with 5 clients connected to a gig switch to a single GigE connected server WinXP client writing 300MB file over a GigE link to a Win2k Server: 6MB/s WinXP client reading 300MB file over a GigE link from a Win2k Server: 14MB/s MacOSX running the Dave Client writing a 300MB file over a GigE link to a FAS960: 18MB/s MacOSX running the Dave Client reading a 300MB file over a GigE link from a FAS960: 23MB/s
My analysis is the Wintel box cannot handle the load at GigE speeds. I haven't had a chance to try using a TCP Offload Engine NIC which would probably help.
-----Original Message----- From: Tim Driggers [mailto:timd@birthdayexpress.com] Sent: Friday, April 02, 2004 11:59 AM To: NetApp List (E-mail) Subject: CIFS performance Issue
Hi All,
I have been playing with an F760 with ONTAP 6.4.1 for about thirty days. I am having a problem understanding a CIFS transfer rate difference I am seeing between writing to and from the NetApp.
First, a little background:
The F760 has a X1025B gigabit card (SX fiber) in it, connected to an Extreme BlackDiamond 6808. The BlackDiamond does -not- have jumbo frames enabled at this time. The NetApp has full flow control configured.
I am able to copy a single (30GB) file (generated by mkfile in FreeBSD) to and from the NetApp over NFSv3 at between 92-96mbit consistently, with the FreeBSD using an NFS mount having a 100mbit link.
I am able to copy the same 30GB file from the NetApp to a Windows 2000 server using CIFS at 130-160mbit consistently. The Win2k server has a gigabit SX card. When copying the file to the NetApp from the Win2k server, I am seeing a maximum throughput of 69mbit, and usually it settles around 33-40mbit. I am monitoring this transfer rate using 'sh port utillization' on the BlackDiamond. I have tested this on three different Win2k boxes, with nearly identical results. I have also followed instructions on NetApp's website on how to 'improve' performance through registry edits, with no apparent result (positive or negative). I have tried three different flowcontrol settings on the Win2k cards (send, recieve and full) - again with no change in performance.
I'm thrilled with the NFS performance, but we are a windows shop. I need to get performance increased on the CIFS writes to the NetApp if it's possible.
I have tried every tweak I could find, and nothing seems to change the rate at which I can write to the NetApp via CIFS. I have included some NetApp information:
slot 8: Gigabit Ethernet Controller II e8 MAC Address: 00:03:47:25:36:6a (auto-1000sx-fd-up) slot 9: NVRAM Memory Size: 32 MB
netapp1> options cifs cifs.audit.autosave.file.extension cifs.audit.autosave.file.limit 0 cifs.audit.autosave.onsize.enable off cifs.audit.autosave.onsize.threshold 500k cifs.audit.autosave.ontime.enable off cifs.audit.autosave.ontime.interval cifs.audit.enable off cifs.audit.file_access_events.enable on cifs.audit.logon_events.enable on cifs.audit.logsize 524288 cifs.audit.saveas /etc/log/adtlog.evt cifs.bypass_traverse_checking on cifs.comment cifs.guest_account cifs.home_dir cifs.home_dir_namestyle ntname cifs.home_dirs_public_for_admin on cifs.idle_timeout 1800 cifs.max_mpx 50 cifs.netbios_aliases cifs.netbios_over_tcp.enable on cifs.nfs_root_ignore_acl on cifs.oplocks.enable off cifs.oplocks.opendelta 8 cifs.per_client_stats.enable off cifs.perm_check_ro_del_ok on cifs.perm_check_use_gid on cifs.restrict_anonymous.enable off cifs.save_case on cifs.scopeid cifs.search_domains cifs.show_snapshot off cifs.shutdown_msg_level 2 cifs.sidcache.enable on cifs.sidcache.lifetime 1440 cifs.snapshot_file_folding.enable off cifs.symlinks.cycleguard on cifs.symlinks.enable on cifs.tcp_window_size 64240 cifs.trace_dc_connection off cifs.trace_login off cifs.wins_servers <edited>
netapp1> options ip ip.fastpath.enable on ip.ipsec.enable off ip.match_any_ifaddr on ip.path_mtu_discovery.enable on ip.ping_throttle.alarm_interval 0 ip.ping_throttle.drop_level 10 ip.tcp.newreno.enable on ip.tcp.sack.enable off
If anyone can help, I would really appreciate it. If I need to supply more information, please just ask.
Thanks,
Tim
This email and its contents may be confidential. If it is and you are not the intended recipient, please do not disclose or use the information within this email or its attachments. If you have received this email in error, please delete it immediately. Thank you.
the volume onto / from which you read / write is composed of at least 6 disks, right ?
Tim Driggers wrote:
Hi Jack,
I am using the same win2k server to copy to/from the NetApp and to/from another Win2k server. The only difference is which interface is used (the Win2k server has 100mbit and GigE to different vlans). I am actually transferring a 30 gigabyte file, not 30 megabyte. Your suggestion to defragment the Win2k box was valid, as the test server was slightly fragmented. I did a quick analysis after the defrag and this is what I found:
Win2k server writing 30GB file over a 100mbit link to a Win2k Server: 12.2MB/sec (link saturated) Win2k server reading 30GB file over a 100mbit link from a Win2k Server: 12.13MB/sec (link saturated) Win2k server writing 30GB file over a GigE link to a NetApp 760: 7MB/sec Win2k server reading 30GB file over a GigE link from a NetApp 760: 25MB/sec
note: these are the best readings after playing with several Win2k gigE settings (including your suggestion of TCP offload). I guess my question is, should I expect better performance from the NetApp?
-----Original Message----- From: Jack Lyons [mailto:jack.lyons@martinagency.com] Sent: Friday, April 02, 2004 9:43 AM To: 'Tim Driggers'; 'NetApp List (E-mail)' Subject: RE: CIFS performance Issue
There are a lot of variables involved.
What client are you using for copying the file to and from the NetApp. Are you using the Win2k server you are comparing it to?
I think some of what is going is that the Win2k maybe caching the writes to the file, but when writing to the Filer it is not caching the reads (or if it is - it isn't doing it efficiently).
I bet if you looked at the defrag map of the Win2k box it would be moderately fragmented.
Also, for a Gig link, a 30 MB file is not that big. Try using a 300 MB file.
We looked at several different solutions using different protocols and CIFS clients. Here is a snapshot of my results.
Single Client WinXP client writing 300MB file over a GigE link to a Win2k Server: 40MB/s WinXP client reading 300MB file over a GigE link from a Win2k Server: 26MB/s MacOSX running the Dave Client writing a 300MB file over a GigE link to a FAS960: 22MB/s MacOSX running the Dave Client reading a 300MB file over a GigE link from a FAS960: 38MB/s
However with 5 clients connected to a gig switch to a single GigE connected server WinXP client writing 300MB file over a GigE link to a Win2k Server: 6MB/s WinXP client reading 300MB file over a GigE link from a Win2k Server: 14MB/s MacOSX running the Dave Client writing a 300MB file over a GigE link to a FAS960: 18MB/s MacOSX running the Dave Client reading a 300MB file over a GigE link from a FAS960: 23MB/s
My analysis is the Wintel box cannot handle the load at GigE speeds. I haven't had a chance to try using a TCP Offload Engine NIC which would probably help.
-----Original Message----- From: Tim Driggers [mailto:timd@birthdayexpress.com] Sent: Friday, April 02, 2004 11:59 AM To: NetApp List (E-mail) Subject: CIFS performance Issue
Hi All,
I have been playing with an F760 with ONTAP 6.4.1 for about thirty days. I am having a problem understanding a CIFS transfer rate difference I am seeing between writing to and from the NetApp.
First, a little background:
The F760 has a X1025B gigabit card (SX fiber) in it, connected to an Extreme BlackDiamond 6808. The BlackDiamond does -not- have jumbo frames enabled at this time. The NetApp has full flow control configured.
I am able to copy a single (30GB) file (generated by mkfile in FreeBSD) to and from the NetApp over NFSv3 at between 92-96mbit consistently, with the FreeBSD using an NFS mount having a 100mbit link.
I am able to copy the same 30GB file from the NetApp to a Windows 2000 server using CIFS at 130-160mbit consistently. The Win2k server has a gigabit SX card. When copying the file to the NetApp from the Win2k server, I am seeing a maximum throughput of 69mbit, and usually it settles around 33-40mbit. I am monitoring this transfer rate using 'sh port utillization' on the BlackDiamond. I have tested this on three different Win2k boxes, with nearly identical results. I have also followed instructions on NetApp's website on how to 'improve' performance through registry edits, with no apparent result (positive or negative). I have tried three different flowcontrol settings on the Win2k cards (send, recieve and full) - again with no change in performance.
I'm thrilled with the NFS performance, but we are a windows shop. I need to get performance increased on the CIFS writes to the NetApp if it's possible.
I have tried every tweak I could find, and nothing seems to change the rate at which I can write to the NetApp via CIFS. I have included some NetApp information:
slot 8: Gigabit Ethernet Controller II e8 MAC Address: 00:03:47:25:36:6a (auto-1000sx-fd-up) slot 9: NVRAM Memory Size: 32 MB
netapp1> options cifs cifs.audit.autosave.file.extension cifs.audit.autosave.file.limit 0 cifs.audit.autosave.onsize.enable off cifs.audit.autosave.onsize.threshold 500k cifs.audit.autosave.ontime.enable off cifs.audit.autosave.ontime.interval cifs.audit.enable off cifs.audit.file_access_events.enable on cifs.audit.logon_events.enable on cifs.audit.logsize 524288 cifs.audit.saveas /etc/log/adtlog.evt cifs.bypass_traverse_checking on cifs.comment cifs.guest_account cifs.home_dir cifs.home_dir_namestyle ntname cifs.home_dirs_public_for_admin on cifs.idle_timeout 1800 cifs.max_mpx 50 cifs.netbios_aliases cifs.netbios_over_tcp.enable on cifs.nfs_root_ignore_acl on cifs.oplocks.enable off cifs.oplocks.opendelta 8 cifs.per_client_stats.enable off cifs.perm_check_ro_del_ok on cifs.perm_check_use_gid on cifs.restrict_anonymous.enable off cifs.save_case on cifs.scopeid cifs.search_domains cifs.show_snapshot off cifs.shutdown_msg_level 2 cifs.sidcache.enable on cifs.sidcache.lifetime 1440 cifs.snapshot_file_folding.enable off cifs.symlinks.cycleguard on cifs.symlinks.enable on cifs.tcp_window_size 64240 cifs.trace_dc_connection off cifs.trace_login off cifs.wins_servers <edited>
netapp1> options ip ip.fastpath.enable on ip.ipsec.enable off ip.match_any_ifaddr on ip.path_mtu_discovery.enable on ip.ping_throttle.alarm_interval 0 ip.ping_throttle.drop_level 10 ip.tcp.newreno.enable on ip.tcp.sack.enable off
If anyone can help, I would really appreciate it. If I need to supply more information, please just ask.
Thanks,
Tim
This email and its contents may be confidential. If it is and you are not the intended recipient, please do not disclose or use the information within this email or its attachments. If you have received this email in error, please delete it immediately. Thank you.
-
kemal.kurban -
Stephane Bentebba -
Tim Driggers