toasters

toasters@lists.teaparty.net

2 participants
13529 discussions

RE: clustering filers
by Link, Bill (OTS-EDH) 04 Mar '02

04 Mar '02

We run an 840c cluster with 3TB of storage as part of an AIX HACMP cluster. Prior to that the filer heads were 760's. Although we have had no failures to speak of, during testing we deliberately failed everything from the host rs6000 systems to the filer heads including all network hardware without a problem. (Well, we did have a problem with nfs timeouts during cluster failover because the application couldn't handle the error but we fixed that by making the nfs mounts hard.) Bill Link System Administrator DST Output inc. bill_link(a)billing.com -----Original Message----- From: Jeff Kennedy [mailto:jlkennedy@amcc.com] Sent: Monday, March 04, 2002 7:05 AM To: NDMP List Subject: clustering filers A quick question regarding NetApp filer clutering. How well does it work? Would you spend the money again? Problems you've encountered that make it of questionable value. We are putting more and more applications on the filer that are considered 24X7 when the filer itself is not. Granted, they are extremely reliable but the fact is that they have several single points of failure. I am looking at clustering filers as a way of fixing that. The other possibility is Veritas ServPoint HA on SAN. Thanks. -- ===================== Jeff Kennedy Unix Administrator AMCC jlkennedy(a)amcc.com Notice: This e-mail and any attachments are intended only for the individual or company to which it is addressed and may contain information which is privileged, confidential and prohibited from disclosure or unauthorized use under applicable law. If you are not the intended recipient of this e-mail, you are hereby notified that any use, dissemination, or copying of this e-mail or the information contained in this e-mail is strictly prohibited by the sender. If you have received this transmission in error, please return the material received to the sender and delete all copies from your system. Thank you.

2 1

RE: Reset of Archive bit causing very large backups
by Adrian Phillips 04 Mar '02

04 Mar '02

Hi Robert; We were having the same issue with our backups (Also using Veritas, though it was/is Backup Exec 8.5). The solution is to upgrade to 6.1R1P1D9 (or later). Bug # is 49892. The other suggestion / solution is through the Veritas s/w, with the suggestion of changing the job from incremental/differential to Daily or Working Set, which would back up files changed in the last x days. Cheers, -Adrian ps - our filer is an F760... -----Original Message----- From: Wiebke, Robert (Robert) [mailto:wiebke@lucent.com] Sent: Tuesday, February 19, 2002 9:08 AM To: Toaster List Subject: Reset of Archive bit causing very large backups Hello Toasters, We are running 5.3.6r1 on our main Filer 740 and we have noticed very large differential backups lately. A full backup will take up approx. 340GB of tape, however our differential backups are taking up on average 80GB. Last week we saw two consecutive differential backups of 240GB and 340GB. I've spoken to Veritas (Using Datacenter 3.4 with a W2K master server) and they tell me the backups themselves are accurate. While investigating this we noticed that files not accessed in a long time (6+ months) were still showing their archive bits set. As these files have not been changed in sometime the archive bit would be expected to not be set. Sure enough, on all files we looked at, the archive bit was set on all. Is this a bug? Is this documented and is a fix available? I've opened a case with the NOW site, and as of yet I have not heard from them. Would you expect an 80GB differental backup on a 340GB filer? Even on weekends when less data was being accessed? Any comments appreciated. Thanks!

2 1

Webmail farm
by Jose Celestino 04 Mar '02

04 Mar '02

Hi all. We are currently experiencing some heavy load on a filer serving as storage to a webmail farm: FILER> sysstat 1 [...] 63% 5583 0 0 1047 4996 3524 16 0 0 3 70% 6002 0 0 999 6005 3836 0 0 0 3 65% 5738 0 0 1067 5829 2671 0 0 0 3 68% 5881 0 0 972 6195 3424 16 0 0 3 83% 7174 0 0 1363 7401 5477 0 0 0 3 88% 7951 0 0 1609 8026 3984 0 0 0 3 91% 8041 0 0 1387 8357 7076 16 0 0 3 87% 7732 0 0 1369 8508 4601 0 0 0 3 87% 7258 0 0 1196 7554 6006 681 0 0 3 100% 6290 0 0 1039 6406 8108 5108 0 0 3 95% 6953 0 0 1381 6488 7536 2783 0 0 3 88% 8205 0 0 1427 8375 5456 0 0 0 3 73% 6115 0 0 993 6408 5051 16 0 0 3 79% 7046 0 0 1138 7779 2629 0 0 0 3 83% 6851 0 0 1181 7212 8240 0 0 0 3 86% 7888 0 0 1417 8185 5305 16 0 0 3 79% 7435 0 0 1217 7646 1676 0 0 0 3 50% 4001 0 0 664 4293 2490 0 0 0 3 48% 4253 0 0 711 3939 1564 16 0 0 3 46% 4115 0 0 681 4066 1265 0 0 0 3 [...] The farm consists of 6 frontends, 2xPentiumIII - 800Mhz, 1Gb ram, 100Mb Fast Ethernet, running apache and an altered c-client, doing direct maildir access (no imap, direct filesystem access). The frontends go to about 250 concurrent sessions. There are nearly 1 million (1000000) maildir stored in here. FILER> version NetApp Release 6.0.1R2: Fri Feb 9 01:12:44 PST 2001 FILER> ifconfig -a e0: flags=848043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.103 netmask 0xffffff00 broadcast 192.168.1.255 partner inet 192.168.1.104 (not in use) ether 00:a0:98:00:9f:0a (100tx-fd-up) e2a: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d4 (auto-unknown-cfg_down) e2b: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d5 (auto-unknown-cfg_down) e2c: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d6 (auto-unknown-cfg_down) e2d: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d7 (auto-unknown-cfg_down) e7: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:03:47:22:85:5e (auto-1000sx-fd-down) flowcontrol full lo: flags=948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 4056 inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1 ether 00:00:00:00:00:00 (Shared memory) The filer volume webmail: FILER> df Filesystem kbytes used avail capacity Mounted on /vol/webmail/ 406736736 383407488 23329248 94% /vol/webmail/ /vol/webmail/.snapshot 101684180 0 101684180 0% /vol/webmail/.snapshot is mounted in each of the 6 frontends. NFSstat gives me: FILER> nfsstat Server rpc: TCP: calls badcalls nullrecv badlen xdrcall 0 0 0 0 0 UDP: calls badcalls nullrecv badlen xdrcall 350325996760 0 0 0 Server nfs: calls badcalls 393275655620 Server nfs V2: (25634001711 calls) null getattr setattr root lookup readlink read 0 0% 2872301897 11%41557694 0%0 0% 5182588465 20%124663 0% 16772257357 65% wrcache write create remove rename link symlink 0 0% 604329011 2%7689267 0% 19040366 0%12918825 0%11991858 0%26947 0% mkdir rmdir readdir statfs 167656 0% 827086 0% 108179901 0%718 0% Server nfs V3: (13693563851 calls) null getattr setattr lookup access readlink read 0 0% 5446218 0% 43552013 0%2360996181 17%28976674 0%97560 0% 10689673307 78% write create mkdir symlink mknod remove rmdir 410587211 3%5298393 0% 161919 0% 4 0% 0 0% 15451799 0%82070 0% rename link readdir readdir+ fsstat fsinfo pathconf 12977697 0%9969929 0% 110291020 1%0 0% 928 0% 928 0% 0 0% commit 0 0% The getattr seems way too big and this may point to a bad caching on the frontends. But could this bring the CPU to 100% most of the time? Could this be a wafl issue related with the low available space on the volume? I'll first increase the nfs client cache to try to lower the getattr's. But I fear this won't help much, further optimization, from the bottom up, is needed. Any ideas to help optimize the performance in this scenario? Any ideas are welcome. If you need any further info (I wanted to send a filestats but is taking an eternity...) please ask. TIA. -- Jose Celestino <japc(a)co.sapo.pt> SysAdmin::SAPO.pt http://www.sapo.pt --------------------------------------------------------------------- main(){printf("%xu%xk%x!\n",15,12,237);}

4 6

RE: Webmail farm
by Aldrich David S 04 Mar '02

04 Mar '02

Jose, Try getting your filesystem under 90%. Strange things start happening to the filer when filesystems get above 92-94% full. -----Original Message----- From: Jose Celestino [mailto:japc@co.sapo.pt] Sent: Monday, March 04, 2002 11:41 AM To: toasters(a)mathworks.com Subject: Webmail farm Hi all. We are currently experiencing some heavy load on a filer serving as storage to a webmail farm: FILER> sysstat 1 [...] 63% 5583 0 0 1047 4996 3524 16 0 0 3 70% 6002 0 0 999 6005 3836 0 0 0 3 65% 5738 0 0 1067 5829 2671 0 0 0 3 68% 5881 0 0 972 6195 3424 16 0 0 3 83% 7174 0 0 1363 7401 5477 0 0 0 3 88% 7951 0 0 1609 8026 3984 0 0 0 3 91% 8041 0 0 1387 8357 7076 16 0 0 3 87% 7732 0 0 1369 8508 4601 0 0 0 3 87% 7258 0 0 1196 7554 6006 681 0 0 3 100% 6290 0 0 1039 6406 8108 5108 0 0 3 95% 6953 0 0 1381 6488 7536 2783 0 0 3 88% 8205 0 0 1427 8375 5456 0 0 0 3 73% 6115 0 0 993 6408 5051 16 0 0 3 79% 7046 0 0 1138 7779 2629 0 0 0 3 83% 6851 0 0 1181 7212 8240 0 0 0 3 86% 7888 0 0 1417 8185 5305 16 0 0 3 79% 7435 0 0 1217 7646 1676 0 0 0 3 50% 4001 0 0 664 4293 2490 0 0 0 3 48% 4253 0 0 711 3939 1564 16 0 0 3 46% 4115 0 0 681 4066 1265 0 0 0 3 [...] The farm consists of 6 frontends, 2xPentiumIII - 800Mhz, 1Gb ram, 100Mb Fast Ethernet, running apache and an altered c-client, doing direct maildir access (no imap, direct filesystem access). The frontends go to about 250 concurrent sessions. There are nearly 1 million (1000000) maildir stored in here. FILER> version NetApp Release 6.0.1R2: Fri Feb 9 01:12:44 PST 2001 FILER> ifconfig -a e0: flags=848043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.103 netmask 0xffffff00 broadcast 192.168.1.255 partner inet 192.168.1.104 (not in use) ether 00:a0:98:00:9f:0a (100tx-fd-up) e2a: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d4 (auto-unknown-cfg_down) e2b: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d5 (auto-unknown-cfg_down) e2c: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d6 (auto-unknown-cfg_down) e2d: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:20:fc:1e:63:d7 (auto-unknown-cfg_down) e7: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500 ether 00:03:47:22:85:5e (auto-1000sx-fd-down) flowcontrol full lo: flags=948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 4056 inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1 ether 00:00:00:00:00:00 (Shared memory) The filer volume webmail: FILER> df Filesystem kbytes used avail capacity Mounted on /vol/webmail/ 406736736 383407488 23329248 94% /vol/webmail/ /vol/webmail/.snapshot 101684180 0 101684180 0% /vol/webmail/.snapshot is mounted in each of the 6 frontends. NFSstat gives me: FILER> nfsstat Server rpc: TCP: calls badcalls nullrecv badlen xdrcall 0 0 0 0 0 UDP: calls badcalls nullrecv badlen xdrcall 350325996760 0 0 0 Server nfs: calls badcalls 393275655620 Server nfs V2: (25634001711 calls) null getattr setattr root lookup readlink read 0 0% 2872301897 11%41557694 0%0 0% 5182588465 20%124663 0% 16772257357 65% wrcache write create remove rename link symlink 0 0% 604329011 2%7689267 0% 19040366 0%12918825 0%11991858 0%26947 0% mkdir rmdir readdir statfs 167656 0% 827086 0% 108179901 0%718 0% Server nfs V3: (13693563851 calls) null getattr setattr lookup access readlink read 0 0% 5446218 0% 43552013 0%2360996181 17%28976674 0%97560 0% 10689673307 78% write create mkdir symlink mknod remove rmdir 410587211 3%5298393 0% 161919 0% 4 0% 0 0% 15451799 0%82070 0% rename link readdir readdir+ fsstat fsinfo pathconf 12977697 0%9969929 0% 110291020 1%0 0% 928 0% 928 0% 0 0% commit 0 0% The getattr seems way too big and this may point to a bad caching on the frontends. But could this bring the CPU to 100% most of the time? Could this be a wafl issue related with the low available space on the volume? I'll first increase the nfs client cache to try to lower the getattr's. But I fear this won't help much, further optimization, from the bottom up, is needed. Any ideas to help optimize the performance in this scenario? Any ideas are welcome. If you need any further info (I wanted to send a filestats but is taking an eternity...) please ask. TIA. -- Jose Celestino <japc(a)co.sapo.pt> SysAdmin::SAPO.pt http://www.sapo.pt --------------------------------------------------------------------- main(){printf("%xu%xk%x!\n",15,12,237);}

1 0

Re: Webmail farm
by Jose Celestino 04 Mar '02

04 Mar '02

Thus spake Probert, Simon - CLM, on Tue, Mar 05, 2002 at 07:12:21AM +1300: > Disk space and Network transport appear to be the most likely. Use Statit > to gather information about volume performance. Consider going to GB > Ethernet. This can be done on copper instead of fiber saving some > implementation costs. > But won't an increase in bandwidth result in a higher CPU load as more packets will get to/from the filer for processing ? > Simon Probert > > Team Leader - EAS > > Mobile 025 484 676 > > Ext. 65303 > > > > -----Original Message----- > From: Jose Celestino [mailto:japc@co.sapo.pt] > Sent: Tuesday, 5 March 2002 6:41 a.m. > To: toasters(a)mathworks.com > Subject: Webmail farm > > > Hi all. > > We are currently experiencing some heavy load on a filer serving as storage > to a webmail farm: [cut] -- Jose Celestino <japc(a)co.sapo.pt> SysAdmin::SAPO.pt http://www.sapo.pt --------------------------------------------------------------------- main(){printf("%xu%xk%x!\n",15,12,237);}

1 0

(no subject)
by Wollermann, Brian 04 Mar '02

04 Mar '02

1 0

clustering filers
by Jeff Kennedy 04 Mar '02

04 Mar '02

A quick question regarding NetApp filer clutering. How well does it work? Would you spend the money again? Problems you've encountered that make it of questionable value. We are putting more and more applications on the filer that are considered 24X7 when the filer itself is not. Granted, they are extremely reliable but the fact is that they have several single points of failure. I am looking at clustering filers as a way of fixing that. The other possibility is Veritas ServPoint HA on SAN. Thanks. -- ===================== Jeff Kennedy Unix Administrator AMCC jlkennedy(a)amcc.com

1 0

shelfchk command does not give any output
by Kumar, Rahul 02 Mar '02

02 Mar '02

Hi I am not getting any output out of the command shelfchk ie the options for LEDS glow etc etc. The only message which i get is something like this: filer3> shelfchk No hot swap support detected on this system filer3> Any ideas why should i get it ? Rahul

2 1

application reactions to quotas
by neil lehrer 01 Mar '02

01 Mar '02

hi, have people seen unfriendly application responses to quotas? for example, i just tried to save a file with word2000 that would have execeeded the quota. winword dr. watson'ed. pc is winnt4 sp6a, filer is ontap 6.1.1r2. thanks

1 0

Random Trouble with AMANDA
by Jamin A. Brown 01 Mar '02

01 Mar '02

Hello, Looking for any ideas (or possibly even a fix!) from you fine people about a problem we are having. Background: We recently started backing up our NetAPP 720 Filer using a combination of AMANDA and the scripts located at: http://www.xs4all.nl/~johnpc/amanda-netapp.html The server we are using is a dual PIII 800 running FreeBSD. Due to the size of our tape drive (35GB), we have to perform subtree backups. Due to how the NetApp was setup, we're doing 40 subtree backups from the same volume each night... The Problem: Every night, no matter what, some of the backups fail for various reasons (see below). The thing is, these failures are random. A subtree that had problems one night will not have problems the next... I have indexing turned off, and have replced restore with a wrapper as recommended in the article here: http://groups.google.com/groups?hl=en&frame=right&th=25ab1f922c37dcf3&seekm… FAILURE AND STRANGE DUMP SUMMARY: magneto /dev/moya/home/customers/9 lev 0 FAILED [disk /dev/moya/home/customers/9 offline on magneto?] magneto /dev/moya/home/customers/8 lev 0 FAILED [disk /dev/moya/home/customers/8 offline on magneto?] magneto /dev/moya/home/customers/n lev 0 FAILED [disk /dev/moya/home/customers/n offline on magneto?] magneto /dev/moya/alias lev 0 FAILED [disk /dev/moya/alias offline on magneto?] magneto /dev/moya/home/customers/7 lev 1 FAILED [no estimate] The Solution? If anyone has any pointers on things to try, I would greatly appreciate it. We'd like to be able to fix this and stay able to use subtree dumps, and not have to replace our tape drive (which would require the replacing of the library the drive is in due to some bad things...). Thanks in advance. Jamin Brown GWI Operations

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters