Does anyone have any benchmarks/suggestions for poor performace with gigabit ethernet between Sun enterprise 220R and NetApp 740 with Intel GBII card? I have an open ticket w/Netapp and it's been open forever. These machines are directly attached (private network), and I've tried a mkfile and copying that file back during the day and at night (no load) and it makes no difference.
The performance is ~ 10 MB/s, no matter what.
The Sun is running Solaris 8, with latest GB Ethernet 3.0 drivers and patches. Also, the Sun box's only function is Legato server, so I've even stopped all nsr daemons while the test copy/mkfile is in progress, & it makes no difference.
I've tried NFSv2 vs. NFSv3, with 32k size packets and 8k size packets. I've tried tcp and udp, with the same results.
The /etc/system file on the E220R looks like this, for the ge adapter:
* Increase size of STREAMS syncronized queues to increase network * performance. Release notes for qfe card recommends setting this * to 25 per 64Mbytes of RAM in the system. It also prevents * receive overrun on the GbE interface. msmiley 7/20/00
set sq_max_size=775 set ge:ge_adv_1000autoneg_cap=0 set ge:ge_adv_1000fdx_cap=1 set ge:ge_adv_pauseTX=1
Also, I've set tcp/udp specific stuff per the Netapp tech suppt. folks:
ndd -set /dev/udp udp_recv_hiwat 65535 ndd -set /dev/udp udp_xmit_hiwat 65535 ndd -set /dev/tcp tcp_recv_hiwat 65535 ndd -set /dev/tcp tcp_xmit_hiwat 65535
The netapp is a 740 (5.3.6R1P1) with over 100GB of free space available on the volume I'm copying to/from.
Interestingly enough, I have done a packet trace (pktt) on the GBII card and it looks OK. Also, the throughput from my 100BaseT card to the netapp (thru a catalyst 5500) does ~ 7.5 MB/s.
*Any* input here would be much appreciated.
-Marc
Marcus Smiley | Ericsson Wireless Communications, Inc. Unix Sys Admin | 6210 Spine Rd. Boulder, CO 80301 | Ph (303) 473-6903
On Wed, 6 Sep 2000, Marcus Smiley wrote:
The performance is ~ 10 MB/s, no matter what.
Not much to say here... plugged everything in (even borrowed a couple of old Sun SSA FC-AL cables instead of "real" 1000base-SX), and this is what I get with a dd write (reads were slightly faster, but I didn't keep the sysstat output):
slot 8: Gigabit Ethernet Controller II e8 MAC Address: 00:90:27:fc:50:48 (auto-1000sx-fd-up) Revision: 3
CPU NFS CIFS HTTP Net kB/s Disk kB/s Tape kB/s Cache in out read write read write age 100% 591 0 0 19935 119 132 20616 0 0 >60 100% 563 0 0 18997 114 68 21454 0 0 >60 100% 542 0 0 18306 110 44 23415 0 0 >60 100% 600 0 0 20263 121 78 19563 0 0 >60 [...]
Client is a 4x450-MHz E420R, Solaris 2.6. Filer is an F740 (7x36GB drives) with the Intel Gigabit II NIC in slot 8. Checksum offloading gave a 5% increase, if that. Adding seven more drives didn't improve the throughput, as I had hoped. NFSv3 over UDP, 32K read/write block sizes. No tuning done on the Solaris side. Both were plugged into the gigabit uplink ports on a Cisco Catalyst 3548.
-
Brian Tao -
Marcus Smiley