toasters November 2000

toasters@lists.teaparty.net

95 participants
94 discussions

Finding who is pounding your NetApp
by John Stoffel 09 Nov '00

09 Nov '00

Hi all, Just to drag this conversation back to a purely NetApp, purely NFS scenario, I'd like to get some help and pointers on how I can solve a problem I had this morning in a more general and useful way. Let me give you the background details here. We have a bunch of toasters here, various old F330s, an F520 (soon to be retired) and some F740s. This morning a bunch of people were complaining that their workstations were slow, that home directories were timing out, etc. These people all had their home directories on an F330 running OnTap 5.2.1. It has 192Mb of RAM, four shelves, each with 7 x 4gb disks. The poor system was simply pinned to the wall by a client. The CPU was hovering between 85% and 100%, it was reading and writing around 2.3Mb/s to the disks constantly. The nfsstats told me that about 23% of the traffic was writes, the rest was attr lookups and reads. The usual mix of NFS traffic. The cache age was down around 4-5 (it's normally much higher), so I knew it was getting hit hard with writes. But since the system was on a direct link back to a switch, and since I don't run the network at all and don't have access to it, I couldn't tell which system(s) were beating it up. We ended up putting in a PC on a repeater to sniff the link between the switch and the NetApp to try and figure out which host(s) was the bad boy. Once we figured that out, it still didn't help since the two hosts didn't look loaded at all, nor were there any runaway processes sucking up IO that I could find. The clients were both quad processor Suns running Solaris 2.5.1 or 2.6. I use the following tools to try and figure out what was going on here, and failed. We had to reboot the two systems to solve the problem. Now as a Unix admin, this really pained me, since I should have been able to find the culprits and just kill them off. We used these on the solaris side: snoop tcpdump lsof top ps (in all kinds of variations). ethereal (found after the fact, will be used in the future). And on the NetApp side I used: nfsstat netstat -n netstat -r sysstat 4 And while they all showed me something, none of them could show me what I needed. On the NetApp side I needed something to show me the top 10 NFS hosts but IP address, but I couldn't get it to work. The output of 'netstat -r' wasn't a help at all. On the Solaris side, tcpdump showed me the traffic, but didn't give me a way to relate it back to a specific process. And while lsof showed me processes, it didn't show me which one was writing data and at what rate. Does anyone have any hints? I'm thinking of upgrading to 5.3.6 at some point, just to bring the F330s upto date with the F740s, but I'm not in a rush really. Ideally, something on the NetApp side to show me the top NFS clients in terms of Data Rate, or anything would be a god-send. Then something on the Client side to figure out which process(es) were the NFS hogs would also be good. Thanks, John John Stoffel - Senior Unix Systems Administrator - Lucent Technologies stoffel(a)lucent.com - http://www.lucent.com - 978-952-7548

7 6

Re: Finding who is pounding your NetApp
by Ketil Kirkerud Elgethun 08 Nov '00

08 Nov '00

John Stoffel <stoffel(a)casc.com> writes: > On the NetApp side I needed something to show me the top 10 NFS hosts > but IP address, but I couldn't get it to work. The output of 'netstat > -r' wasn't a help at all. nfsstat -l will give you (almost) what you need. For 'nfsstat -l' to work you'll have to enable it first, though. (options nfs.per_client_stats.enable on) nfsstat -l will give you nfs ops per host. A small script to do something like #!/bin/sh rsh filer nfsstat -z sleep 60 rsh filer nfsstat -l (or -h) will give you the traffic (in nfs ops) for all clients active during the last 60 seconds. You might want to sort the output, though. Have a look at the manpage (available in /etc/man on your filer) for na_nfsstat for more info. (running with nfs.per_client_stats.enable on will possibly impair performance slightly, you might want to enable it just for this purpose, then disable it when you don't need it. Does anyone know how much of a performance hit this could generate ?) regards, -- ---Ketil Kirkerud Elgethun, Infostream ASP Marcus Thranesgate 6, N-0473 Oslo Phone: +47 22 70 45 00, Direct: +47 22 70 47 16, Mobile: +47 934 11 394

2 1

250K NFS ops/sec, eh?
by Brian Tao 08 Nov '00

08 Nov '00

http://www.spec.org/osg/sfs97/results/res2000q3/sfs97-20000905-00200.html What's special about a "scalable storage cluster", or is that just a new marketing name given to a bunch of independent F840's? A quarter million ops per second... sweet. :) -- Brian Tao (BT300, taob(a)risc.org) "Though this be madness, yet there is method in't"

3 5

RE: Exchange 2k (long)
by Whitfield, Charlie 08 Nov '00

08 Nov '00

Best be honest Peter why don't you use your usual email address. ;-) FYI, Peter can be contacted at work on bodifee_peter(a)emc.com (if I'm not mistaken). Cheers, your old friend from a previous life, CAW Charlie Whitfield Network Appliance UK email: caw(a)netapp.com voice: +44 20 8756 6762 mobile: +44 7718 160954 -----Original Message----- From: Peter Bodifee [mailto:bodif36772@yahoo.com] Sent: 08 November 2000 14:50 To: 'Bercovici, Val'; supak.lailert(a)yipintsoi.com Cc: toasters(a)mathworks.com; 'Martin Hannigan'; rdobbins(a)netmore.net Subject: RE: Exchange 2k (long) Dear Val, Please be honest and open to the user community and add " and has no plans to do so in the future" to the second sentence. See http://support.microsoft.com/support/kb/articles/q250/3/48.asp Peter -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com]On Behalf Of Bercovici, Val Sent: dinsdag 7 november 2000 21:14 To: 'supak.lailert(a)yipintsoi.com' Cc: toasters(a)mathworks.com; Martin Hannigan; rdobbins(a)netmore.net Subject: RE: Exchange 2k (long) Hi Supak et al, Please let me clarify your support statement below. Microsoft does not currently support NAS for Exchange Server 5.5 or 2000. Network Appliance fully supports Exchange 5.5 on filers via our SnapManager 1.x for Exchange software. We have engaged IBM Global Services' highly-regarded Enterprise Services for Microsoft Technologies Practice for our escalation path regarding Exchange-specific issues that we cannot resolve. [ ... marketing stuff removed for clarity ...] Thanks, -Val. Manager, Collaborative Strategy Network Appliance Inc. WHO STORES MORE THAN HALF THE WORLD'S E-MAILBOXES? FIND OUT AT: www.netapp.com 613.724.8674 - Phone 847.589.6510 - Fax 800.489.6924 - Pager E-Mail: valb(a)netapp.com -----Original Message----- From: Supak backup e-mail account [mailto:supaklailert@yahoo.com] Sent: Friday, November 03, 2000 10:31 PM To: Martin Hannigan; rdobbins(a)netmore.net Cc: toasters(a)mathworks.com Subject: Re: Exchange 2k NetApp filer is fully supported for use with MS Exchange. I don't see how this could lead to a problem more than any direct-attaced storage. NetApp choking on lots of small files? I don't think so. WAFL handles small files far better than most file systems notably UFS. Waste of space? Maybe. NetApp uses 4K blocks so if you have lots of really small files <4K then you'll waste quite a bit of space. Supak --- Martin Hannigan <hannigan(a)world.std.com> wrote: > > > > > > For both performance and supportability reasons, > it is extremely inadvisable > > to put Exchange mailboxes/private stores/public > stores on anything other > > than a filesystem which is directly-connected to > the NT server in question. > > > > People do all sorts of odd things with NetApps > just because they can - and > > don't get me wrong, I love my F740 - but this is > just plain crazy. For your > > own peace of mind, don't do it. > > > So the netapp still chokes on lots of active small > files i.e. like > usenet news, etc? We were thinking about turning > ours into a hardcore > news spool..but I remember a few years ago there was > some issue with the > millions of < 64KB file sizes.. > > -M > __________________________________________________ Do You Yahoo!? Thousands of Stores. Millions of Products. All in one Place. http://shopping.yahoo.com/ _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com

1 0

RE: Exchange 2k (not very long)
by mds＠gbnet.net 08 Nov '00

08 Nov '00

*ding*ding* End of round two. Before we go much further, can anyone explain what DaFS or however it's spelt might change the situation, if at all? > Please be honest and introduce yourself to the others as an employee of EMC. Teehee. > Please be honest and open to the user community and add " and has no plans > to do so in the future" to the second sentence. Nothing dishonest in leaving out information - I mean we're deep into the mire of marketecture here as it is. Caveat Emptor all Ye Who Enter Here, or something. > See http://support.microsoft.com/support/kb/articles/q250/3/48.asp Very well justified though. I believe anyone who wants Exchange, and wants the advantages of NetApp or any other decently managed storage is not going down a dark and perilous alley. In fact they'll probably be getting better support from all the vendors involved than most Unix email system customers ever get from any vendors. But whether you or whether you don't want to use them together, well, gee, is it really worth the bruises arguing about it? -- -Mark ... an Englishman in Derby ...

1 0

RE: Exchange 2k (long)
by Traitel, Eyal 08 Nov '00

08 Nov '00

Dear Peter, Please be honest and introduce yourself to the others as an employee of EMC. Microsoft is making business on selling Windows NT/2000 as file serving as well, and in this case, it is a competitor to the NAS market, hence its running away from supporting it. It does not support also any specific driver a customer might use, which all of us use everyday, so support declaration are "touchy" issues always. This is also not a secret that Network Appliance filers are performing more than well on CIFS benchmarks, and it is a competition here too. It is interesting too to see that anyway EMC is trying to imitate our well regarded Exchange-related products as well, but no market presence yet... Eyal. ---------------------------------------------------------------------- eTraitel - I'm the new eBuzzword around !!! ---------------------------------------------------------------------- Eyal Traitel - Filer Escalation Engineer CNA, MCSE, CSA, NACA Network Appliance BV Holland Office Center Kruisweg 799b 2132 NG, Hoofddorp The Netherlands Office: +31 23 567 9685 Cellular: +31 6 5497 2568 Email: eyal(a)netapp.com ---------------------------------------------------------------------- Get answers NOW! - NetApp On the Web - http://now.netapp.com ---------------------------------------------------------------------- -----Original Message----- From: Peter Bodifee [mailto:bodif36772@yahoo.com] Sent: Wednesday, November 08, 2000 3:50 PM To: 'Bercovici, Val'; supak.lailert(a)yipintsoi.com Cc: toasters(a)mathworks.com; 'Martin Hannigan'; rdobbins(a)netmore.net Subject: RE: Exchange 2k (long) Dear Val, Please be honest and open to the user community and add " and has no plans to do so in the future" to the second sentence. See http://support.microsoft.com/support/kb/articles/q250/3/48.asp Peter -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com]On Behalf Of Bercovici, Val Sent: dinsdag 7 november 2000 21:14 To: 'supak.lailert(a)yipintsoi.com' Cc: toasters(a)mathworks.com; Martin Hannigan; rdobbins(a)netmore.net Subject: RE: Exchange 2k (long) Hi Supak et al, Please let me clarify your support statement below. Microsoft does not currently support NAS for Exchange Server 5.5 or 2000. Network Appliance fully supports Exchange 5.5 on filers via our SnapManager 1.x for Exchange software. We have engaged IBM Global Services' highly-regarded Enterprise Services for Microsoft Technologies Practice for our escalation path regarding Exchange-specific issues that we cannot resolve. [ ... marketing stuff removed for clarity ...] Thanks, -Val. Manager, Collaborative Strategy Network Appliance Inc. WHO STORES MORE THAN HALF THE WORLD'S E-MAILBOXES? FIND OUT AT: www.netapp.com 613.724.8674 - Phone 847.589.6510 - Fax 800.489.6924 - Pager E-Mail: valb(a)netapp.com -----Original Message----- From: Supak backup e-mail account [mailto:supaklailert@yahoo.com] Sent: Friday, November 03, 2000 10:31 PM To: Martin Hannigan; rdobbins(a)netmore.net Cc: toasters(a)mathworks.com Subject: Re: Exchange 2k NetApp filer is fully supported for use with MS Exchange. I don't see how this could lead to a problem more than any direct-attaced storage. NetApp choking on lots of small files? I don't think so. WAFL handles small files far better than most file systems notably UFS. Waste of space? Maybe. NetApp uses 4K blocks so if you have lots of really small files <4K then you'll waste quite a bit of space. Supak --- Martin Hannigan <hannigan(a)world.std.com> wrote: > > > > > > For both performance and supportability reasons, > it is extremely inadvisable > > to put Exchange mailboxes/private stores/public > stores on anything other > > than a filesystem which is directly-connected to > the NT server in question. > > > > People do all sorts of odd things with NetApps > just because they can - and > > don't get me wrong, I love my F740 - but this is > just plain crazy. For your > > own peace of mind, don't do it. > > > So the netapp still chokes on lots of active small > files i.e. like > usenet news, etc? We were thinking about turning > ours into a hardcore > news spool..but I remember a few years ago there was > some issue with the > millions of < 64KB file sizes.. > > -M > __________________________________________________ Do You Yahoo!? Thousands of Stores. Millions of Products. All in one Place. http://shopping.yahoo.com/ _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com

1 0

RE: Sharing Filers among different user groups
by Clawson, Simon 08 Nov '00

08 Nov '00

We run an F760 here, and it is accessed by two different groups. The data held by one group cannot be accessed by another. This is achived by using CIFS Shares under NT, and by using the UNIX file permission on the directories themselves, and by setting the NFS export up to have the correct Anon UID. You may need to alter your infrestructure to achive what you want though. We have made the whole sharing scenario possible by implementing it in a single NFS environment. We have one NIS master that servers the whole office, along with local slaves in each department. The only area where the Netapps has a single access point is in the NT domain, where it has to bind to a domain controller on one of the three subnets we have. All you need to do in this situation is establish trust relationships between the other NT subnets and map the Netapps accross these. Bit confusing all that..but it is do-able... Hope that helps Simon >With the advent of the F840, we are thinking of doing some massive server >consolidation: using a F840 to replace many existing nfs/cifs servers among >different user groups. >What are the issues with sharing a filer among different user groups? I can >see: >1) uid issues: Dept A is already using uid 101 for user smith, and Dept B >has a different smith, or has a 101 for a different users ... etc. >2) security issues: we want to make absolutely sure Dept A cannot see or >mount or share Dept B's data. >3) what if Dept A is using a different DNS than Dept B's? I don't think we >can run 2 different resolvers, one per exported volumes ... >3) similarly, for NIS, and NT domains. >Any insights, comments are appreciated. Xt _________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com. Share information about yourself, create your own public profile at http://profiles.msn.com.

1 0

Veritas NetBackup backup hangs
by Jeffrey Krueger 08 Nov '00

08 Nov '00

We have recently cut-over to using Veritas NetBackup with NDMP to perform a mixture of backups to filer-attached-tape and 3-way backups to other filers with tape. We are running DOT 5.3.4R3P2 on a mixture of 630's and 760's against NBU 3.2 with a recent patch set (one that includes the 3-way fix). In both instance, we have found that sometimes a filer completes a dump (ndmp session goes away, backup snapshot is deleted, /etc/dumpdates is updates), but the backup job hangs in Veritas-land. This job hangs indefiately until manually killed, at which point it does not record the file history making the backup un-restore-able. We're growing very frustrated with this issue. BudTool has been backing these filers up for 2 years with NDMP and relatively few problems. So far, NBU support has been disappointing to say the least. Has anyone seen this? Is there a workaround or fix? Also, does anyone know the mechanics of the "/etc/ndmpdlog.*" files? They are poorly documented in the DOT manuals. All of ours are zero length, but its unclear whether this is supposed to be the case (no news is good news). Does anyone know what they are supposed to look like? -- Jeff -- ---------------------------------------------------------------------------- Jeff Krueger E-Mail: jeff(a)qualcomm.com Senior Engineer Phone: 858-651-6709 NetApp Filers / UNIX Infrastructure Fax: 858-651-6627 QUALCOMM, Inc. IT Engineering Web: www.qualcomm.com

1 0

Looking for customers who want to use FrontPage and/or pass-throu gh authentication with filers
by Benn, Paul 08 Nov '00

08 Nov '00

Hi Gang, I'm looking for a few customers who are interested in using FrontPage server extensions (now called "Office 2000 server extensions") and/or IIS pass-through authentication with filers. Please reply privately. Regards, Paul Benn Network Appliance

1 0

rsh return code
by Brian Tao 08 Nov '00

08 Nov '00

Probably a dumb question, but I just had a gigantic Sunday brunch buffet, so my brain is halfway into a food coma. ;-) Is it possible to have rsh return 0 or 1 depending on the success of a command executed on a filer? For instance, I want my backup scripts to kick off a "snap create" before they proceed, but I want to check the exit status of the rsh so I have a chance to abort and notify the operator if the snapshot is not created. -- Brian Tao (BT300, taob(a)risc.org) "Though this be madness, yet there is method in't"

4 5

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters November 2000