Not sure what version of Data OnTap you are running but have you tried
running netdiag to show any lower layer errors. Also, try running
sysstat -s 1 to show "real-time" performance stats. We evaluated a FAS880
initially and I think the CIFS protocol is more CPU intensive than NFS and
when the CPU >99% we started dropping a lot of packets and saw crappy
numbers.
-----Original Message-----
From: Tim Driggers [mailto:timd@birthdayexpress.com]
Sent: Sunday, April 04, 2004 8:58 PM
To: 'Jack Lyons'; Tim Driggers; 'NetApp List (E-mail)'
Subject: RE: CIFS performance Issue
Hi Jack,
I am using the same win2k server to copy to/from the NetApp and to/from
another Win2k server. The only difference is which interface is used (the
Win2k server has 100mbit and GigE to different vlans). I am actually
transferring a 30 gigabyte file, not 30 megabyte. Your suggestion to
defragment the Win2k box was valid, as the test server was slightly
fragmented. I did a quick analysis after the defrag and this is what I
found:
Win2k server writing 30GB file over a 100mbit link to a Win2k Server:
12.2MB/sec (link saturated)
Win2k server reading 30GB file over a 100mbit link from a Win2k Server:
12.13MB/sec (link saturated)
Win2k server writing 30GB file over a GigE link to a NetApp 760: 7MB/sec
Win2k server reading 30GB file over a GigE link from a NetApp 760: 25MB/sec
note: these are the best readings after playing with several Win2k gigE
settings (including your suggestion of TCP offload). I guess my question
is, should I expect better performance from the NetApp?
-----Original Message-----
From: Jack Lyons [mailto:jack.lyons@martinagency.com]
Sent: Friday, April 02, 2004 9:43 AM
To: 'Tim Driggers'; 'NetApp List (E-mail)'
Subject: RE: CIFS performance Issue
There are a lot of variables involved.
What client are you using for copying the file to and from the NetApp. Are
you using the Win2k server you are comparing it to?
I think some of what is going is that the Win2k maybe caching the writes to
the file, but when writing to the Filer it is not caching the reads (or if
it is - it isn't doing it efficiently).
I bet if you looked at the defrag map of the Win2k box it would be
moderately fragmented.
Also, for a Gig link, a 30 MB file is not that big. Try using a 300 MB
file.
We looked at several different solutions using different protocols and CIFS
clients. Here is a snapshot of my results.
Single Client
WinXP client writing 300MB file over a GigE link to a Win2k Server: 40MB/s
WinXP client reading 300MB file over a GigE link from a Win2k Server: 26MB/s
MacOSX running the Dave Client writing a 300MB file over a GigE link to a
FAS960: 22MB/s
MacOSX running the Dave Client reading a 300MB file over a GigE link from a
FAS960: 38MB/s
However with 5 clients connected to a gig switch to a single GigE connected
server
WinXP client writing 300MB file over a GigE link to a Win2k Server: 6MB/s
WinXP client reading 300MB file over a GigE link from a Win2k Server: 14MB/s
MacOSX running the Dave Client writing a 300MB file over a GigE link to a
FAS960: 18MB/s
MacOSX running the Dave Client reading a 300MB file over a GigE link from a
FAS960: 23MB/s
My analysis is the Wintel box cannot handle the load at GigE speeds.
I haven't had a chance to try using a TCP Offload Engine NIC which would
probably help.
-----Original Message-----
From: Tim Driggers [mailto:timd@birthdayexpress.com]
Sent: Friday, April 02, 2004 11:59 AM
To: NetApp List (E-mail)
Subject: CIFS performance Issue
Hi All,
I have been playing with an F760 with ONTAP 6.4.1 for about thirty days. I
am having a problem understanding a CIFS transfer rate difference I am
seeing between writing to and from the NetApp.
First, a little background:
The F760 has a X1025B gigabit card (SX fiber) in it, connected to an Extreme
BlackDiamond 6808. The BlackDiamond does -not- have jumbo frames enabled at
this time. The NetApp has full flow control configured.
I am able to copy a single (30GB) file (generated by mkfile in FreeBSD) to
and from the NetApp over NFSv3 at between 92-96mbit consistently, with the
FreeBSD using an NFS mount having a 100mbit link.
I am able to copy the same 30GB file from the NetApp to a Windows 2000
server using CIFS at 130-160mbit consistently. The Win2k server has a
gigabit SX card. When copying the file to the NetApp from the Win2k server,
I am seeing a maximum throughput of 69mbit, and usually it settles around
33-40mbit. I am monitoring this transfer rate using 'sh port utillization'
on the BlackDiamond. I have tested this on three different Win2k boxes, with
nearly identical results. I have also followed instructions on NetApp's
website on how to 'improve' performance through registry edits, with no
apparent result (positive or negative). I have tried three different
flowcontrol settings on the Win2k cards (send, recieve and full) - again
with no change in performance.
I'm thrilled with the NFS performance, but we are a windows shop. I need to
get performance increased on the CIFS writes to the NetApp if it's possible.
I have tried every tweak I could find, and nothing seems to change the rate
at which I can write to the NetApp via CIFS. I have included some NetApp
information:
slot 8: Gigabit Ethernet Controller II
e8 MAC Address: 00:03:47:25:36:6a (auto-1000sx-fd-up)
slot 9: NVRAM
Memory Size: 32 MB
netapp1> options cifs
cifs.audit.autosave.file.extension
cifs.audit.autosave.file.limit 0
cifs.audit.autosave.onsize.enable off
cifs.audit.autosave.onsize.threshold 500k
cifs.audit.autosave.ontime.enable off
cifs.audit.autosave.ontime.interval
cifs.audit.enable off
cifs.audit.file_access_events.enable on
cifs.audit.logon_events.enable on
cifs.audit.logsize 524288
cifs.audit.saveas /etc/log/adtlog.evt
cifs.bypass_traverse_checking on
cifs.comment
cifs.guest_account
cifs.home_dir
cifs.home_dir_namestyle ntname
cifs.home_dirs_public_for_admin on
cifs.idle_timeout 1800
cifs.max_mpx 50
cifs.netbios_aliases
cifs.netbios_over_tcp.enable on
cifs.nfs_root_ignore_acl on
cifs.oplocks.enable off
cifs.oplocks.opendelta 8
cifs.per_client_stats.enable off
cifs.perm_check_ro_del_ok on
cifs.perm_check_use_gid on
cifs.restrict_anonymous.enable off
cifs.save_case on
cifs.scopeid
cifs.search_domains
cifs.show_snapshot off
cifs.shutdown_msg_level 2
cifs.sidcache.enable on
cifs.sidcache.lifetime 1440
cifs.snapshot_file_folding.enable off
cifs.symlinks.cycleguard on
cifs.symlinks.enable on
cifs.tcp_window_size 64240
cifs.trace_dc_connection off
cifs.trace_login off
cifs.wins_servers <edited>
netapp1> options ip
ip.fastpath.enable on
ip.ipsec.enable off
ip.match_any_ifaddr on
ip.path_mtu_discovery.enable on
ip.ping_throttle.alarm_interval 0
ip.ping_throttle.drop_level 10
ip.tcp.newreno.enable on
ip.tcp.sack.enable off
If anyone can help, I would really appreciate it. If I need to supply more
information, please just ask.
Thanks,
Tim
This email and its contents may be confidential. If it is and you are not
the intended recipient, please do not disclose or use the information within
this email or its attachments. If you have received this email in error,
please delete it immediately. Thank you.
This email and its contents may be confidential. If it is and you are not
the intended recipient, please do not disclose or use the information within
this email or its attachments. If you have received this email in error,
please delete it immediately. Thank you.