toasters

toasters@lists.teaparty.net

2 participants
13529 discussions

Re: gigabit ethernet performance
by Darrell Fuhriman 07 Sep '00

07 Sep '00

Premanshu Jain <PrJain(a)shastanets.com> writes: > Oh..That's even better. We got a max of 8 mbytes/sec. Never more than that. > Interestingly, we get the same throughput between any sun boxes as well... No offense, but you must be doing something wrong. Our Suns (E450s) can do at least 60MB/second over gigabit. Darrell

1 0

RE: gigabit ethernet performance
by Premanshu Jain 07 Sep '00

07 Sep '00

Oh..That's even better. We got a max of 8 mbytes/sec. Never more than that. Interestingly, we get the same throughput between any sun boxes as well... prem -----Original Message----- From: Marcus Smiley [mailto:marcus.smiley@ericsson.com] Sent: Wednesday, September 06, 2000 2:41 PM To: toasters(a)mathworks.com Subject: gigabit ethernet performance Does anyone have any benchmarks/suggestions for poor performace with gigabit ethernet between Sun enterprise 220R and NetApp 740 with Intel GBII card? I have an open ticket w/Netapp and it's been open forever. These machines are directly attached (private network), and I've tried a mkfile and copying that file back during the day and at night (no load) and it makes no difference. The performance is ~ 10 MB/s, no matter what. The Sun is running Solaris 8, with latest GB Ethernet 3.0 drivers and patches. Also, the Sun box's only function is Legato server, so I've even stopped all nsr daemons while the test copy/mkfile is in progress, & it makes no difference. I've tried NFSv2 vs. NFSv3, with 32k size packets and 8k size packets. I've tried tcp and udp, with the same results. The /etc/system file on the E220R looks like this, for the ge adapter: * Increase size of STREAMS syncronized queues to increase network * performance. Release notes for qfe card recommends setting this * to 25 per 64Mbytes of RAM in the system. It also prevents * receive overrun on the GbE interface. msmiley 7/20/00 set sq_max_size=775 set ge:ge_adv_1000autoneg_cap=0 set ge:ge_adv_1000fdx_cap=1 set ge:ge_adv_pauseTX=1 Also, I've set tcp/udp specific stuff per the Netapp tech suppt. folks: ndd -set /dev/udp udp_recv_hiwat 65535 ndd -set /dev/udp udp_xmit_hiwat 65535 ndd -set /dev/tcp tcp_recv_hiwat 65535 ndd -set /dev/tcp tcp_xmit_hiwat 65535 The netapp is a 740 (5.3.6R1P1) with over 100GB of free space available on the volume I'm copying to/from. Interestingly enough, I have done a packet trace (pktt) on the GBII card and it looks OK. Also, the throughput from my 100BaseT card to the netapp (thru a catalyst 5500) does ~ 7.5 MB/s. *Any* input here would be much appreciated. -Marc Marcus Smiley | Ericsson Wireless Communications, Inc. Unix Sys Admin | 6210 Spine Rd. Boulder, CO 80301 | Ph (303) 473-6903

1 0

Re: gigabit ethernet performance
by Marcus Smiley 07 Sep '00

07 Sep '00

no there is no switch between the filer and Sun box, using crossover fiber jumper scenario. yes, both machines are set to FULL flow control. Here's a sysstat 1 before, during & after I create a 100MB file, using mkfile: CPU NFS CIFS HTTP Net kB/s Disk kB/s Tape kB/s Cache in out read write read write age 3% 52 0 0 19 43 0 0 0 0 38 3% 65 2 0 11 10 0 0 0 0 38 4% 35 2 0 51 13 0 0 0 0 38 3% 6 23 0 167 10 0 0 0 0 38 4% 36 23 0 14 5 0 0 0 0 38 4% 29 79 0 72 28 52 0 0 0 38 38% 10 260 0 22 57 4412 10316 0 0 38 6% 20 2 0 36 136 3 0 0 0 38 59% 601 1 0 13398 374 0 0 0 0 38 82% 398 16 0 9009 291 4684 14874 0 0 38 73% 572 23 0 12229 363 2972 7208 0 0 38 87% 457 2 0 10112 326 3688 16404 0 0 38 80% 327 0 0 9509 284 5980 14176 0 0 38 73% 520 15 0 14158 405 1093 5053 0 0 38 93% 342 0 0 8070 269 7148 18589 0 0 38 68% 356 2 0 9640 283 5428 11073 0 0 38 82% 591 0 0 11667 346 4112 9208 0 0 38 88% 562 3 0 8520 506 6740 15836 0 0 38 32% 135 1 0 4585 271 1077 4032 0 0 38 3% 19 0 0 4 4 0 0 0 0 38 CPU NFS CIFS HTTP Net kB/s Disk kB/s Tape kB/s Cache in out read write read write age 3% 64 0 0 3 2 0 0 0 0 38 3% 48 0 0 14 12 0 0 0 0 38 3% 17 2 0 13 10 0 0 0 0 38 3% 4 12 0 30 5 0 0 0 0 38 3% 4 12 0 4 2 0 0 0 0 38 On Wed, 06 Sep 2000, Paul Albers wrote: ! Marcus, ! ! Do you have flow control enabled on both Rx and Tx on the ! port the filer is connected to on the switch? If not, try enabling ! it and retest. How does the filer look during writes and ! reads ( sysstat 1 )? Also, what is the configuration of the ! filer ( cards in what slots? ). ! ! Regards, ! ! -paul ! ! Marcus Smiley wrote: ! ! > slot 8 ! > ! > On Wed, 06 Sep 2000, Larry Armstrong wrote: ! > ! > ! Howdy! ! > ! ! > ! Out of curiousity, what slot do you have your GbE card in? ! > ! ! > ! ---larry >:) ! > ! _ ! > ! * Marcus Smiley (marcus.smiley(a)ericsson.com) done spit this rhetoric: ! > ! > Does anyone have any benchmarks/suggestions for poor performace with ! > ! > gigabit ethernet between Sun enterprise 220R and NetApp 740 with Intel ! > ! > GBII card? I have an open ticket w/Netapp and it's been open ! > ! > forever. These machines are directly attached (private network), and ! > ! > I've tried a mkfile and copying that file back during the day and at ! > ! > night (no load) and it makes no difference. ! > ! > ! > ! > The performance is ~ 10 MB/s, no matter what. ! > ! > ! > ! > The Sun is running Solaris 8, with latest GB Ethernet 3.0 drivers ! > ! > and patches. Also, the Sun box's only function is Legato server, ! > ! > so I've even stopped all nsr daemons while the test copy/mkfile ! > ! > is in progress, & it makes no difference. ! > ! > ! > ! > I've tried NFSv2 vs. NFSv3, with 32k size packets and 8k size packets. ! > ! > I've tried tcp and udp, with the same results. ! > ! > ! > ! > The /etc/system file on the E220R looks like this, for the ge adapter: ! > ! > ! > ! > * Increase size of STREAMS syncronized queues to increase network ! > ! > * performance. Release notes for qfe card recommends setting this ! > ! > * to 25 per 64Mbytes of RAM in the system. It also prevents ! > ! > * receive overrun on the GbE interface. msmiley 7/20/00 ! > ! > ! > ! > set sq_max_size=775 ! > ! > set ge:ge_adv_1000autoneg_cap=0 ! > ! > set ge:ge_adv_1000fdx_cap=1 ! > ! > set ge:ge_adv_pauseTX=1 ! > ! > ! > ! > Also, I've set tcp/udp specific stuff per the Netapp tech suppt. folks: ! > ! > ! > ! > ndd -set /dev/udp udp_recv_hiwat 65535 ! > ! > ndd -set /dev/udp udp_xmit_hiwat 65535 ! > ! > ndd -set /dev/tcp tcp_recv_hiwat 65535 ! > ! > ndd -set /dev/tcp tcp_xmit_hiwat 65535 ! > ! > ! > ! > The netapp is a 740 (5.3.6R1P1) with over 100GB of free space available on the ! > ! > volume I'm copying to/from. ! > ! > ! > ! > Interestingly enough, I have done a packet trace (pktt) on the GBII ! > ! > card and it looks OK. Also, the throughput from my 100BaseT card to ! > ! > the netapp (thru a catalyst 5500) does ~ 7.5 MB/s. ! > ! > ! > ! > *Any* input here would be much appreciated. ! > ! > ! > ! > -Marc ! > ! > ! > ! > ! > ! > ! > ! > Marcus Smiley | Ericsson Wireless Communications, Inc. ! > ! > Unix Sys Admin | 6210 Spine Rd. Boulder, CO 80301 | Ph (303) 473-6903 ! > ! > ! > ! ! > ! -- ! > ! Larry Armstrong ! > ! <surfer(a)netapp.com> ! > ! ! > ! bovarism n. - A magnified opinion of one's own abilities. ! > ! > Marcus Smiley | Ericsson Wireless Communications, Inc. ! > Unix Sys Admin | 6210 Spine Rd. Boulder, CO 80301 | Ph (303) 473-6903 ! ! -- ! "Nothing will benefit human health and increase the chances of survival ! of life on earth as much as the evolution to a vegetarian diet." ! ! -Albert Einstein !

1 0

Re: gigabit ethernet performance
by Marcus Smiley 07 Sep '00

07 Sep '00

slot 8 On Wed, 06 Sep 2000, Larry Armstrong wrote: ! Howdy! ! ! Out of curiousity, what slot do you have your GbE card in? ! ! ---larry >:) ! _ ! * Marcus Smiley (marcus.smiley(a)ericsson.com) done spit this rhetoric: ! > Does anyone have any benchmarks/suggestions for poor performace with ! > gigabit ethernet between Sun enterprise 220R and NetApp 740 with Intel ! > GBII card? I have an open ticket w/Netapp and it's been open ! > forever. These machines are directly attached (private network), and ! > I've tried a mkfile and copying that file back during the day and at ! > night (no load) and it makes no difference. ! > ! > The performance is ~ 10 MB/s, no matter what. ! > ! > The Sun is running Solaris 8, with latest GB Ethernet 3.0 drivers ! > and patches. Also, the Sun box's only function is Legato server, ! > so I've even stopped all nsr daemons while the test copy/mkfile ! > is in progress, & it makes no difference. ! > ! > I've tried NFSv2 vs. NFSv3, with 32k size packets and 8k size packets. ! > I've tried tcp and udp, with the same results. ! > ! > The /etc/system file on the E220R looks like this, for the ge adapter: ! > ! > * Increase size of STREAMS syncronized queues to increase network ! > * performance. Release notes for qfe card recommends setting this ! > * to 25 per 64Mbytes of RAM in the system. It also prevents ! > * receive overrun on the GbE interface. msmiley 7/20/00 ! > ! > set sq_max_size=775 ! > set ge:ge_adv_1000autoneg_cap=0 ! > set ge:ge_adv_1000fdx_cap=1 ! > set ge:ge_adv_pauseTX=1 ! > ! > Also, I've set tcp/udp specific stuff per the Netapp tech suppt. folks: ! > ! > ndd -set /dev/udp udp_recv_hiwat 65535 ! > ndd -set /dev/udp udp_xmit_hiwat 65535 ! > ndd -set /dev/tcp tcp_recv_hiwat 65535 ! > ndd -set /dev/tcp tcp_xmit_hiwat 65535 ! > ! > The netapp is a 740 (5.3.6R1P1) with over 100GB of free space available on the ! > volume I'm copying to/from. ! > ! > Interestingly enough, I have done a packet trace (pktt) on the GBII ! > card and it looks OK. Also, the throughput from my 100BaseT card to ! > the netapp (thru a catalyst 5500) does ~ 7.5 MB/s. ! > ! > *Any* input here would be much appreciated. ! > ! > -Marc ! > ! > ! > ! > Marcus Smiley | Ericsson Wireless Communications, Inc. ! > Unix Sys Admin | 6210 Spine Rd. Boulder, CO 80301 | Ph (303) 473-6903 ! > ! ! -- ! Larry Armstrong ! <surfer(a)netapp.com> ! ! bovarism n. - A magnified opinion of one's own abilities. Marcus Smiley | Ericsson Wireless Communications, Inc. Unix Sys Admin | 6210 Spine Rd. Boulder, CO 80301 | Ph (303) 473-6903

1 0

quad ethernet -> gigabit on Cisco switches
by Todd C. Merrill 06 Sep '00

06 Sep '00

We are considering moving our F740 and F760 toasters from quad 100-Mbit ethernet EtherChannel'ed on a Cisco 5500 switch to the gigabit NICs on a new Cisco switch, a 6509. We're doing this 'cause: 1. We need more ports. Period. 2. The 6509 has a higher density of ports and gig ports. 3. The gigabit NICs are supposedly much faster, and you can offload the checksum processing from the main CPU to the gigabit card. (We're not even close to saturating the quad link now; I'm just talking latencies. Plus, we beat the crap out of our toasters; if the CPU can do less overhead and more data serving, I want it.) Anybody have any experience with the 6509 and NetApp's gigabit cards? Just want to assuage our concerns before placing the PO's. Until next time... The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill(a)mathworks.com http://www.mathworks.com ---

3 2

Re: quad ethernet -> gigabit on Cisco switches
by Rick Hulsey 06 Sep '00

06 Sep '00

We use the F-760 in a cluster and Cisco 6509 switches with Gigabit connectivity to Suns and HP-UX boxes. We haven't had any problems with Gig-e or the switches, but we haven't got them spun up to full production usage levels yet. I would say that you need to make sure that you get the Intel NICs for the filer, but if you are order new gear now from Netapp, that should be a given. Rick Hulsey Southwest Airlines 2425 Wyman Mail Stop 4DC Dallas, Tx., 75235 rick.hulsey(a)wnco.com (214) 792-7188 Office (972) 880-6882 Cell (800) 915-3747 Pager >>> "Todd C. Merrill" <tmerrill(a)mathworks.com> 09/06/00 10:14AM >>> We are considering moving our F740 and F760 toasters from quad 100-Mbit ethernet EtherChannel'ed on a Cisco 5500 switch to the gigabit NICs on a new Cisco switch, a 6509. We're doing this 'cause: 1. We need more ports. Period. 2. The 6509 has a higher density of ports and gig ports. 3. The gigabit NICs are supposedly much faster, and you can offload the checksum processing from the main CPU to the gigabit card. (We're not even close to saturating the quad link now; I'm just talking latencies. Plus, we beat the crap out of our toasters; if the CPU can do less overhead and more data serving, I want it.) Anybody have any experience with the 6509 and NetApp's gigabit cards? Just want to assuage our concerns before placing the PO's. Until next time... The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill(a)mathworks.com http://www.mathworks.com ---

1 0

RE: ESTALE
by Strange, Steve 06 Sep '00

06 Sep '00

Deletion-handling of open files in NFS V2 and V3 (such as it is) is handled at the NFS client. The NFS server isn't aware of which clients have a file open, or how many processes on those clients have a particular file open (I'm assuming no NFS locking here). If a process on NFS client "A" has a file open, then another process on that same client deletes the file, the NFS client sends a "RENAME" to the NFS server, instead of a "REMOVE", so that the file sticks around until that first process closes the file (at least this is the case for NFS clients with which I'm familiar). The file is renamed to ".nfsXXXX", or something of that ilk, but the NFS file handle remains the same, so the first process can continue to read/write/etc. However, if a process on a different NFS client, "B", deletes the file, client "B" has no way of knowing that the file is open by a process on client "A". So client "B" will go ahead and remove the file completely, resulting in ESTALE the next time the process on client "A" attemts an operation on the file. Do you have just one NFS client accessing the file in question? What NFS client are you using? Hope this chatter is of some use. Cheers, Steve > -----Original Message----- > From: Edward Hibbert [mailto:EH@dataconnection.com] > Sent: Wednesday, September 06, 2000 4:39 AM > To: toasters(a)mathworks.com > Subject: ESTALE > > > Does anyone know under precisely what circumstances NetApp > returns ESTALE? > > I have a file open in my application to which I have just > done a write; I > issue another write very shortly thereafter and I get ESTALE back. > > Now, I believed that ESTALE was returned when a file handle > was presented > where the file had really been deleted and the inode for the > file was reused > for a new file. But since my process has the file open, even > if it's been > deleted by some other process (which is possible), I don't > think I should > get ESTALE returned. > > So this means I don't understand what is going on. But > perhaps you do :-). > > > Edward Hibbert. >

1 0

NetApp 9gb FC/AL drives available for sale
by ScottUCE＠aol.com 06 Sep '00

06 Sep '00

A leasing company has SEVEN 9gb FC/AL fiber disk drives (NetApp Model X219F) available; NetApp no longer offers these drives. They are asking $4,500.00 for the seven drives, shelf/power supply and cabling. Please contact me if you have any interest. Scott Fischmann Union Computer Exchange, Inc. 6233 Idylwood Lane Minneapolis, MN 55436 (612) 935-7282 (612) 935-5056 - FAX www.unioncomputer.com

1 0

ESTALE
by Edward Hibbert 06 Sep '00

06 Sep '00

Does anyone know under precisely what circumstances NetApp returns ESTALE? I have a file open in my application to which I have just done a write; I issue another write very shortly thereafter and I get ESTALE back. Now, I believed that ESTALE was returned when a file handle was presented where the file had really been deleted and the inode for the file was reused for a new file. But since my process has the file open, even if it's been deleted by some other process (which is possible), I don't think I should get ESTALE returned. So this means I don't understand what is going on. But perhaps you do :-). Edward Hibbert.

1 0

RE: Budtool woes
by Clawson, Simon 05 Sep '00

05 Sep '00

Hi! Thanks for the info! Scary stuff this backup! I have just checked the logs to find that the backup ran for 32 hours this weekend! Anyway, How have you achived the spliting of your file system? I think this is going to cause us problems due to the rather unitelligent autoloader we use to drive the backup...any advise on this? Simon -----Original Message----- From: Marion Hakanson [mailto:hakanson@cse.ogi.edu] Sent: 01 September 2000 18:24 To: Clawson, Simon Cc: toasters(a)mathworks.com Subject: Re: Budtool woes > . . . > I am bacing up a single volume filer with nearly 200GB of data - my backup > is constanlty failing due to the process running out of time..anyone seen > this? Is there a way round it? Can I increse the allowable time? Can I split > the backup into several levels? > . . . Simon, I see that you've probably solved your timeout problem, but you've probably got another problem as well. I've just upgraded to bt-4.6.1a, and the Release Notes state that it does not support backups that get split across more than 3 tapes. I'll spare you the details right now, since you can go look at my original posting to "toasters" on this topic: http://teaparty.mathworks.com:1999/toasters/6612.html Apparently a large backup of this type will work fine, but you may not be able to do recoveries from it, under certain mysterious circumstances (which I've not yet experienced, thankfully). The workaround is to break up that single backup into backups of subdirectories of that large volume (you don't have to breakup the volume itself). Solaris ufsdump can do this, but doesn't support incremental (non-full) dumps of subdirectories; Fortunately (by the tests I've done here), NetApp's dump (both console/cmd-line and NDMP) can do both fulls and incrementals of subdirectories. I've since noticed that NetApp themselves recommend against backing up large volumes (>100GB) in a single "dump" run. Here's where: http://now.netapp.com/NOW/knowledge/docs/ontap/rel536r2/html/sag/dump7.htm Luckily for us, our large volume (185GB, currently) is broken up into about ten qtrees & a couple other directories. I think the 10-12 dumps actually complete slightly faster than the previous single large dump. BTW, how's it going in your search for a BudTool replacement? A PDC engineer told me that the new Legato Networker 6.0 NDMP support will only do backups to either a NetApp-attached library, or to another NDMP enabled system (and _not_ to a Networker media server). It doesn't support a split-library configuration at all (which is what we have our BudTool/NetApp setup doing). Because of that, they want to sell me Veritas NetBackup instead. I've got a few other leads in the works, and I'd be happy to compare notes on what you've found so far. Good luck and regards, -- Marion Hakanson <hakanson(a)cse.ogi.edu> CSE Computing Facilities

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters