toasters

toasters@lists.teaparty.net

3 participants
13514 discussions

RE: Slooow SGI Challenge
by Miller, James 04 Jan '01

04 Jan '01

Hi Bill: I spent 7 years field service for SGI. You have 2 problems, neither can be fixed. 1)The VME 100Mb (C8-100BT)interface turned out to be a piece of junk even before IRIX 6.5 got hold of it. The board, as you said is only half duplex, a seven year old design that never got a major rework because it's on a long time dead product line. I had several sites that were never able to exceed 65Mb simplex. 2) IRIX 6.5 has made problems and performance worse that under IRIX 6.2 For systems under SGI support, GPS would swap out the C8-100BT board for a FDDI (SAS/DAS) of their choice and Customer would use a CISCO, or alike, to run a FDDI Fibre connection into a 100Mb switch. There are some kernel parameters to tweak for this that I cannot remember for MTU. We did this for 2 customers just in the Phila area alone because of the additional problems IRIX 6.5.x created. FIX, Replace your Challenge with a O2100 and get reliable 100 & 1000Mb FDX interfaces. ... All is my humble opinion, and not that of my previous employer ..jim -----Original Message----- From: Link, Bill (OTS-EDH) [mailto:Bill_Link@billing.com] Sent: Thursday, January 04, 2001 4:57 PM To: 'toasters(a)mathworks.com' Subject: Slooow SGI Challenge Has anyone had any experience with netapp on an SGI Challenge system? I have an F760 attached through a cisco 4000 switch to an sgi challenge and an IBM S7A. I have this arrangement on a dedicated subnet with all interfaces set manually to 100tx. (The vme controller in the SGI will not abide full duplex). The netapp is running 5.3.6R1, the SGI is running IRIX 6.5.5, and the RS6K is running AIX4.3.3. The IBM will create a 128M file in under 30 sec. If I mount the filer on the SGI box and specify tcp it takes about 2 min 45 sec to write the same file. If I let the mount on the SGI default to udp it won't write more that 15 or 20 mb before it starts getting nfs timeouts. The filer seems oblivious to any trouble that the sgi is having. The sgi will work (slowly) at 10mb half duplex with udp. Any suggestions would be much appreciated. Bill Link SysAdmin Output Technology Solutions bill_link(a)billing.com

1 0

Slooow SGI Challenge
by Link, Bill (OTS-EDH) 04 Jan '01

04 Jan '01

Has anyone had any experience with netapp on an SGI Challenge system? I have an F760 attached through a cisco 4000 switch to an sgi challenge and an IBM S7A. I have this arrangement on a dedicated subnet with all interfaces set manually to 100tx. (The vme controller in the SGI will not abide full duplex). The netapp is running 5.3.6R1, the SGI is running IRIX 6.5.5, and the RS6K is running AIX4.3.3. The IBM will create a 128M file in under 30 sec. If I mount the filer on the SGI box and specify tcp it takes about 2 min 45 sec to write the same file. If I let the mount on the SGI default to udp it won't write more that 15 or 20 mb before it starts getting nfs timeouts. The filer seems oblivious to any trouble that the sgi is having. The sgi will work (slowly) at 10mb half duplex with udp. Any suggestions would be much appreciated. Bill Link SysAdmin Output Technology Solutions bill_link(a)billing.com

1 0

Dump dies around 13G bytes
by Fred Ab 04 Jan '01

04 Jan '01

I get DUMP IS ABORTED when trying to dump a 39G byte volume to a DLT 7000 drive directly connected to the NetApp. I know that it is 35G native and 70G compressed but it looks like it dies around 13G. DUMP: creating "/vol/vol1/.snapshot/snapshot_for_backup.109" snapshot. DUMP: Using Partial Volume Dump DUMP: Date of this level 0 dump: Fri Dec 22 23:05:09 2000. DUMP: Date of last level 0 dump: the epoch. DUMP: Dumping /vol/vol1/proj to /dev/nrst0a DUMP: mapping (Pass I)[regular files] DUMP: mapping (Pass II)[directories] DUMP: estimated 38916473 tape blocks on 0.97 tapes DUMP: dumping (Pass III) [directories] DUMP: dumping (Pass IV) [regular files] DUMP: We have written 1585525 KB to tape. DUMP: We have written 3465622 KB to tape. DUMP: We have written 5328897 KB to tape. DUMP: We have written 7627776 KB to tape. DUMP: We have written 10240766 KB to tape. DUMP: DUMP IS ABORTED ultra119# pwd /america/vol1/proj ultra119# df -k . Filesystem kbytes used avail capacity Mounted on america:/vol/vol1 53766696 39259976 14506720 74% /america/vol1 sysconfig -t output: nrst0a - no rewind device, format is: 85937 bpi 70 GB (w/comp) Dump Command = dump 0ubBf 63 39845869 /dev/nrst0a Any ideas why?

1 0

Gotcha on multiple network interfaces
by Brian L. Brush 04 Jan '01

04 Jan '01

If you have multiple network interfaces on your filers, this information may be useful to you. If you're thorough about reading the documentation and clever about checking your results, you may have discovered it for yourself. While testing our new F760 cluster, we have discovered that if the filer has two interfaces on the same subnet and the default gateway resides on that subnet, the default route in the filer's routing table will choose the interface with the lower number. In particular, using the "setup" command, we configured an F760 with on-board 100Mbps e0 and GigE e7 on the same subnet. Using "netstat -r", we determined that the default route was using e0, which was not what we wanted--the 100Mbps was intended to be a backup link, not for regular production use: pow> netstat -r Routing tables Internet: Destination Gateway Flags Refs Use Interface default 135.90.100.254 UG 1 61 e0 127 localhost UG 1 9 lo localhost localhost UH 1 0 lo 135.90.100/24 link#1 UC 0 0 e0 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e0 135.90.100.255 link#1 UHL 1 46 e0 After some experimentation, we settled on editing /etc/rc to configure e0 after both configuring e7 and adding the default route; we also turned off routed: #Auto-generated by setup Wed Dec 20 19:16:53 GMT 2000 hostname pow ifconfig e7 `hostname`-e7 flowcontrol full netmask 255.255.255.0 route add default 135.90.100.254 1 ifconfig e0 `hostname`-e0 mediatype auto netmask 255.255.255.0 #routed on options dns.domainname is.paradyne.com options dns.enable on options nis.domainname is.paradyne.com options nis.enable on savecore exportfs -a nfs on Now we have the results we wanted; all outbound packets should use the gig link, according to netstat: pow> netstat -r Routing tables Internet: Destination Gateway Flags Refs Use Interface default 135.90.100.254 UG 5 2148 e7 127 localhost UG 1 261 lo localhost localhost UH 1 0 lo 135.90.100/24 link#2 UC 0 0 e7 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e7 135.90.100.255 link#2 UHL 1 1794 e7 Looking at the systems we had in production, we realized that, as a result of this oversight, they had been sending all outbound traffic via their fast ethernet interfaces rather than gig! Aaargh! --Brian L. Brush Senior Systems Administrator Paradyne Corporation

1 1

Re: Gotcha on multiple network interfaces
by svawter＠c-cube.com 04 Jan '01

04 Jan '01

What you should use for a failover link is create a vif in single mode with e7 & e0, favoring e7. vif create single f0 e7 e0 vif favor e7 ifconfig f0 hostname flowcontrol full netmask 255.255.255.0 or perhaps: vif create single f0 e7 ifconfig f0 hostname flowcontrol full netmask 255.255.255.0 vif favor e7 vif add f0 e0 Steve Vawter voice: 408-490-5310 fax: 408-490-8615 Staff UNIX Systems Administrator Steve.Vawter(a)C-Cube.COM --> --> From brian(a)paradyne.com Thu Jan 4 09:56:51 2001 --> From: "Brian L. Brush" <brian(a)paradyne.com> --> X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.8 sun4u) --> To: toasters(a)mathworks.com --> Subject: Gotcha on multiple network interfaces --> --> --> If you have multiple network interfaces on your filers, this information --> may be useful to you. If you're thorough about reading the documentation --> and clever about checking your results, you may have discovered it for --> yourself. --> --> While testing our new F760 cluster, we have discovered that if the filer --> has two interfaces on the same subnet and the default gateway resides on --> that subnet, the default route in the filer's routing table will choose --> the interface with the lower number. --> --> In particular, using the "setup" command, we configured an F760 with --> on-board 100Mbps e0 and GigE e7 on the same subnet. Using "netstat -r", --> we determined that the default route was using e0, which was not what we --> wanted--the 100Mbps was intended to be a backup link, not for regular --> production use: --> --> pow> netstat -r --> Routing tables --> --> Internet: --> Destination Gateway Flags Refs Use Interface --> default 135.90.100.254 UG 1 61 e0 --> 127 localhost UG 1 9 lo --> localhost localhost UH 1 0 lo --> 135.90.100/24 link#1 UC 0 0 e0 --> 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e0 --> 135.90.100.255 link#1 UHL 1 46 e0 --> --> After some experimentation, we settled on editing /etc/rc to configure --> e0 after both configuring e7 and adding the default route; we also --> turned off routed: --> --> #Auto-generated by setup Wed Dec 20 19:16:53 GMT 2000 --> hostname pow --> ifconfig e7 `hostname`-e7 flowcontrol full netmask 255.255.255.0 --> route add default 135.90.100.254 1 --> ifconfig e0 `hostname`-e0 mediatype auto netmask 255.255.255.0 --> #routed on --> options dns.domainname is.paradyne.com --> options dns.enable on --> options nis.domainname is.paradyne.com --> options nis.enable on --> savecore --> exportfs -a --> nfs on --> --> Now we have the results we wanted; all outbound packets should use the --> gig link, according to netstat: --> --> pow> netstat -r --> Routing tables --> --> Internet: --> Destination Gateway Flags Refs Use Interface --> default 135.90.100.254 UG 5 2148 e7 --> 127 localhost UG 1 261 lo --> localhost localhost UH 1 0 lo --> 135.90.100/24 link#2 UC 0 0 e7 --> 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e7 --> 135.90.100.255 link#2 UHL 1 1794 e7 --> --> Looking at the systems we had in production, we realized that, as a --> result of this oversight, they had been sending all outbound traffic via --> their fast ethernet interfaces rather than gig! Aaargh! --> --> --Brian L. Brush --> Senior Systems Administrator --> Paradyne Corporation -->

1 0

RE: Gotcha on multiple network interfaces
by Premanshu Jain 04 Jan '01

04 Jan '01

Yes..I also observed the same problem and finally disabled the e0 interface. prem -----Original Message----- From: Brian L. Brush [mailto:brian@paradyne.com] Sent: Thursday, January 04, 2001 7:20 AM To: toasters(a)mathworks.com Subject: Gotcha on multiple network interfaces If you have multiple network interfaces on your filers, this information may be useful to you. If you're thorough about reading the documentation and clever about checking your results, you may have discovered it for yourself. While testing our new F760 cluster, we have discovered that if the filer has two interfaces on the same subnet and the default gateway resides on that subnet, the default route in the filer's routing table will choose the interface with the lower number. In particular, using the "setup" command, we configured an F760 with on-board 100Mbps e0 and GigE e7 on the same subnet. Using "netstat -r", we determined that the default route was using e0, which was not what we wanted--the 100Mbps was intended to be a backup link, not for regular production use: pow> netstat -r Routing tables Internet: Destination Gateway Flags Refs Use Interface default 135.90.100.254 UG 1 61 e0 127 localhost UG 1 9 lo localhost localhost UH 1 0 lo 135.90.100/24 link#1 UC 0 0 e0 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e0 135.90.100.255 link#1 UHL 1 46 e0 After some experimentation, we settled on editing /etc/rc to configure e0 after both configuring e7 and adding the default route; we also turned off routed: #Auto-generated by setup Wed Dec 20 19:16:53 GMT 2000 hostname pow ifconfig e7 `hostname`-e7 flowcontrol full netmask 255.255.255.0 route add default 135.90.100.254 1 ifconfig e0 `hostname`-e0 mediatype auto netmask 255.255.255.0 #routed on options dns.domainname is.paradyne.com options dns.enable on options nis.domainname is.paradyne.com options nis.enable on savecore exportfs -a nfs on Now we have the results we wanted; all outbound packets should use the gig link, according to netstat: pow> netstat -r Routing tables Internet: Destination Gateway Flags Refs Use Interface default 135.90.100.254 UG 5 2148 e7 127 localhost UG 1 261 lo localhost localhost UH 1 0 lo 135.90.100/24 link#2 UC 0 0 e7 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e7 135.90.100.255 link#2 UHL 1 1794 e7 Looking at the systems we had in production, we realized that, as a result of this oversight, they had been sending all outbound traffic via their fast ethernet interfaces rather than gig! Aaargh! --Brian L. Brush Senior Systems Administrator Paradyne Corporation

1 0

RE: Sun photon (fcal) disks on toaster?
by Traitel, Eyal 04 Jan '01

04 Jan '01

Games like this could be fun indeed, but I doubt if our code supports them... Anyway - you can always try to boot a regular PC with our Intel platform floppies... :) Eyal. ---------------------------------------------------------------------- eTraitel - I'm the new eBuzzword around !!! ---------------------------------------------------------------------- Eyal Traitel - Filer Escalation Engineer CNA, MCSE, CSA, NACA Network Appliance BV Holland Office Center Kruisweg 799b 2132 NG, Hoofddorp The Netherlands Office: +31 23 567 9685 Cellular: +31 6 5497 2568 Email: eyal(a)netapp.com ---------------------------------------------------------------------- Get answers NOW! - NetApp On the Web - http://now.netapp.com ---------------------------------------------------------------------- -----Original Message----- From: Jason S. Cash [mailto:cash@udel.edu] Sent: Thursday, January 04, 2001 2:48 PM To: toasters(a)mathworks.com Subject: Sun photon (fcal) disks on toaster? Has anyone successfully hooked Sun photon disk arrays to a netapp? I have a bunch of these sitting around at the moment a f740 that it might be amusing to see how fast they can go (if at all). Management might be willing to buy us some 840s to act as heads for these arrays if it will work :) TIA, Jason /* Jason S. Cash - Systems Programmer IT/Network and Systems Services University of Delaware, Newark Delaware */

1 0

Sun photon (fcal) disks on toaster?
by Jason S. Cash 04 Jan '01

04 Jan '01

Has anyone successfully hooked Sun photon disk arrays to a netapp? I have a bunch of these sitting around at the moment a f740 that it might be amusing to see how fast they can go (if at all). Management might be willing to buy us some 840s to act as heads for these arrays if it will work :) TIA, Jason /* Jason S. Cash - Systems Programmer IT/Network and Systems Services University of Delaware, Newark Delaware */

1 0

Filer + Netscape Messaging Server
by Brian L. Brush 03 Jan '01

03 Jan '01

We are considering using a filer to provide the message stores for our Netscape (soon to be iPlanet) Messaging Server. Of course, neither Netscape (iPlanet) nor NetApp supports such a configuration. Can anyone offer relevant insight or experience? For a couple of years we have been using a similarly unsupported configuration to provide storage space for Rational ClearCase. While we've had no insuperable problems, I don't relish the prospect of hitting a brick wall on those rare occasions when we request support--for the application, not the configuration. --Brian L. Brush Senior Systems Administrator Paradyne Corporation

1 0

Re: Bad Solaris performance over time
by hill＠communitytelephone.com 03 Jan '01

03 Jan '01

On Tue, 2 Jan 2001, Chris Lamb wrote: ->One box handled incoming SMTP and all local delivery (sendmail+procmail) ->and POP, the other handled some unholy number of Majordomo lists. I mean, ->this was with hardware that nowadays people scoff at, running ancient ->software releases and NFS V2 over a single network interface. I find it ->impossible to believe that a 4-way UltraII and an F760 can't handle the ->load. (Yeah, the average mail message with all the crap "modern" mailers ->add on is probably several Kb larger, but that's almost never the real ->bottleneck.) That was supposed to be part of my point, the machine we have can more the adequetly handle the load. :) The stats http://static-content-01.communitytelephone.net/~hill/server_stats show that the machine doesn't breathe hard all day long. It seems that something happens over time and at some point hits critical mass, which causes the performance to become very bad. -> ->Some questions that may be a bit obvious: Is your 2.6 box patched to ->current levels? Yes. Patch revisions are posted at the above URL. ->Why NFS V2? This was a typo, it is really V3. ->How much NVRAM in the filer? 32Mb. The filer weekly_log is at the above URL. ->Is the network bogged down, or the switch misconfigured? We see no errors on the switch, or the servers network interfaces. ->How many network interfaces, of what type/speed? Filer has a Gigabit interface. Solaris servers has 100Mb. You can see the throughput information at the above URL file: nx.se. ->What does snoop/tcpdump/your network sniffer say? I haven't tried this yet. It looked like there was going to be way too much information to sort through. I will gather it the next time the system begins to act up. ->How many disks in the volume that contains /var/mail? 14 ->Is this filer running CIFS & NFS, or is it purely an NFS box? NFS only ->If /var/mail is on the filer, what about home directories (i.e., is ->your mailer bogging doing ".forward" lookups)? home directories are on the netapp as well. Again I must state it is not a general performance problem, but something that happens over time. -> Are you using NIS and/or DNS on the filer? No. ->What version of sendmail/qmail/whatever? Latest smail. ->Are you using procmail for delivery? Why not? :-) Yes. ->Is disk scrubbing kicking in? Are you taking snapshots too frequently? ->Do you have cron jobs walking the filesystem or other network clients ->hitting the filer or the network when these slowdowns are observed on ->the ->mail server? And really, why NFS V2? I appreciate what you are getting at here, but I don't believe that sort of thing is happening. It you look at the stats for the filer the performance of it stays stable through the trouble period with the one server. Also, all other servers continue to perform well. I believe the server stats show that things like these aren't occuring as well. We are using V3 mounts I typo'd my earlier email. ->(I'm not really a smartass, I just play one on the Internet. :-) These ->are just some of the questions I'd look to answer before poking at ->/etc/system. We did do that. We feel stronghly that there is a bug in Solaris or in the interaction between it and the netapp which cause our performance problems to occur. We tried to massage /etc/system parameters to keep from occuring. Again, I must point out our server ran great when we have /var/mail on local storage. We only began to see this problem after moving it to the filer and only after 24 - 30 hours of operation. I have found a bug ID on sunsolve that described similarities to our problem, but we already have the patch that was supposed to fix that bug. I include it here so you can get a feel for the actual problem from someone elses perspective. Even though our problem is not an exact match we do have similarities with it. We also have similarities to a posting on toasters Date: Tue, 10 Aug 1999 08:05:18 -0700 (PDT) From: Nick Christenson <npc(a)sendmail.com> To: toasters(a)mathworks.com Subject: NFS client tuning, problem and resolution. ------------------------ Sun Bug ID 4034003 --------------------------- Some time ago I got email from someone on a system w/ 37,000 users who had problem using mail from it's mailserver, both CPU were very busy and the system had slowed to a crawl at peak times. The systems gets +/- 70K sendmail calls, 130K pop3 calls a day, mail lives ona NFSv3 capable network appliance server. Mail is delivered to the home directories, not to /var/mail. He ran kernel profiling and found crcmp() topping the list: granularity: each sample hit covers 2 byte(s) for 0.12% of 8.65 seconds % cumulative self self total time seconds seconds calls ms/call ms/call name 39.2 3.39 3.39 crcmp [1] 2.8 3.63 0.24 locked_pgcopy [2] 2.5 3.85 0.22 splx [3] 1.8 4.01 0.16 bcopy [4] 1.7 4.16 0.15 mutex_enter [5] 1.7 4.31 0.15 disp_getwork [6] 1.6 4.45 0.14 dnlc_purge_vp [7] 1.5 4.58 0.13 nfs3_access [8] 1.5 4.71 0.13 ross625_vac_segflush [9] 1.3 4.82 0.11 blkclr [10] 1.3 4.93 0.11 idle [11] 1.0 5.02 0.09 mutex_adaptive_enter [12] While the information is not complete, here's what I think happens, all users access the same rnode ("/export/home", mail is delivered to the home directies) a large access cache is built and here's where we serialize:, in nfs3_access() : [common/fs/nfs/nfs3_vnops.c] ........ mutex_enter(&rp->r_statelock); for (acp = rp->r_acc; acp != NULL; acp = acp->next) { /* * Look for an entry by comparing credentials. */ if (crcmp(acp->cred, cr) == 0) { if ((acp->known & acc) == acc) { #ifdef DEBUG nfs3_access_cache_hits++; #endif if ((acp->allowed & acc) == acc) { mutex_exit(&rp->r_statelock); return (0); } mutex_exit(&rp->r_statelock); return (EACCES); } break; } } mutex_exit(&rp->r_statelock); And further down, if we have a cache miss, we do the same all over again. (As we unlock, we need to recheck whether a new entry was added, so the list is rechecked, again holding the r_statelock) Now, that algorithm works fine in the typical case where users access their own files; or whena few users do. It does *not* work when you have 37K users accessing a single file. As long as the node is cached and the kernel doesn't reap kernel memory, the list will grow out of bounds and access to this node will get increasingly slow. This theory was tested by switching to NFSv2 mounts. This fixed the problem. Attached is a program that shows the behaviour somewhat. It calls "access()" on a file (specified on the command line) as many different users. The file must exist on a NFSv3 filesystem, the program must be run as root and the file must be openable as root on the client. One sample run printed this: 0 18860 500 5680 1000 4798 1500 3963 2000 3071 2500 2425 3000 1552 3500 863 and this tells that after running access(file, F_OK) as 4000 different uids, there's a really high penalty for the first users that called access. In this case for uid(0) searching the cache costs 18ms. It seems that this cache should not be allowed to grow out of bounds, currently it's ownly limited by kernel memory. -Work Around- Use NFSv2 mounts on filesystems that are used by lots of users such as "/var/mail". michael.eisler@Eng 1997-02-24 The above aork around is fine as long as the server is not a Solaris 2.5[.1] or 2.6 system. Since the ACL protocol has an ACCESS procedure for NFS V2, a Solaris 2.5 or alter system will also use the ACL cache system. ---------------------- End Sun Bug ID 4034003 ------------------------- We did try using V2 mounts as a test, and we still experienced the problem. --Jamie Hill -> ->-- Chris -> ->-- ->Chris Lamb, Unix Guy ->MeasureCast, Inc. ->503-241-1469 x247 -><skeezics(a)measurecast.com> -> ->

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters