toasters

toasters@lists.teaparty.net

3 participants
13534 discussions

Minimum PSUs required for DS4243 with 24x SAS
by Alexander Griesser 07 Feb '13

07 Feb '13

Hey there, we were just re-cabling some of the power cords on our DS4243 shelfs with 4 PSUs and 24x 15k 600GB SAS. When I power down one of the 4 available PSUs in the DS4243, I get the following messages in system log and autosupports are generated: Wed Feb 6 19:28:15 CET [barricade:ses.status.psError:CRITICAL]: DS4243 (S/N SHJHU000000055F) shelf 2 on channel 0a power error for Power supply 4: critical status; power supply failed. This module is on the rear side of the shelf, at the bottom right. Wed Feb 6 19:28:15 CET [barricade:ses.status.volError:CRITICAL]: DS4243 (S/N SHJHU000000055F) shelf 2 on channel 0a voltage error for Voltage sensor 8: unrecoverable error. This module is on the rear side of the shelf, at the lower right, on the power supply. Wed Feb 6 19:28:15 CET [barricade:ses.psu.powerReqError:CRITICAL]: Not enough power supplies are present in channel 0a disk shelf 2 to satisfy disk drive and shelf power requirements. Wed Feb 6 19:28:15 CET [barricade:monitor.shelf.fault:CRITICAL]: Fault reported on disk storage shelf attached to channel 0a. Please check fans, power supplies, disks, and temperature sensors. Wed Feb 6 19:28:15 CET [barricade:callhome.shlf.fault:error]: Call home for SHELF_FAULT Wed Feb 6 19:28:15 CET [barricade:monitor.shelf.configError:CRITICAL]: Enclosure services has detected an error in access to shelves or shelf configuration 0a. Wed Feb 6 19:28:15 CET [barricade:callhome.config:warning]: Call home for CONFIGURATION_ERROR Wed Feb 6 19:29:00 CET [barricade:monitor.globalStatus.critical:CRITICAL]: Disk shelf fault. Disk shelf configuration error. Wed Feb 6 19:29:11 CET [barricade:ses.status.psInfo:info]: DS4243 (S/N SHJHU000000055F) shelf 2 on channel 0a power supply information for Power supply 4: normal status. Wed Feb 6 19:29:11 CET [barricade:ses.status.volInfo:info]: DS4243 (S/N SHJHU000000055F) shelf 2 on channel 0a voltage information for Voltage sensor 8: normal status. Wed Feb 6 19:29:11 CET [barricade:monitor.shelf.fault.ok:info]: Fault previously reported on disk storage shelf attached to channel 0a has been corrected. Wed Feb 6 19:29:11 CET [barricade:monitor.shelf.configError.ok:info]: Enclosure services-detected configuration error has been corrected on channel 0a. Wed Feb 6 19:30:00 CET [barricade:monitor.globalStatus.ok:info]: The system's global status is normal. Why exactly does it think that there are not enough power supplies present if I just shut down one of them? I was under the assumption that a full SAS shelf can work with 2 PSUs and a full SATA shelf can work with 1 PSU without problems. Am I wrong? Nothing happend to the filers, no takeover, no errors, etc., but the message in the logs puzzles me. Thanks && bye, Alexander Griesser System-Administrator ANEXIA Internetdienstleistungs GmbH Telefon: +43-463-208501-320 Telefax: +43-463-208501-500 E-Mail: ag(a)anexia.at <mailto:ag@anexia.at> Web: http://www.anexia.at <http://www.anexia.at> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

3 3

NetApp 8.0.2 7-mode - update to 8.1.2.
by Rafał Radecki 06 Feb '13

06 Feb '13

Hi All. I have a FAS3210 appliance with NetApp 8.0.2. I would like to update it to 8.1.2. In http://support.netapp.com/ (My AutoSupport) I see there is a "Upgrade Advisor" section. There is a step-by-step howto. But are there any good documents about updating this software? I would like to make as few mistakes as possible, I will do it for the first time ;) Best regards, Rafal Radecki.

3 3

Authentication Issues w/ SCCM talking to filer via CIFS w/ Computer Account
by Ray Van Dolson 05 Feb '13

05 Feb '13

We're having authentication issues with SCCM pointed at a CIFS share on a filer running ONTAP 8.0.x. SCCM uses a domain computer account to authenticate (our filer is also joined to our domain). We've added the computer account at the share level (as well as "Domain Computers") with full permissions, but continue to get authentication denied errors back from the filer. Speficially: Fri Feb 1 16:28:27 PST [red-str-napc2-p2: auth.trace.authenticateUser.loginTraceIP:info]: AUTH: Login attempt by user red-inf-cm-p01$ of domain DOMAIN from client machine 1.1.1.1 (RED-INF-CM-P01). Fri Feb 1 16:28:27 PST [red-str-napc2-p2: auth.dc.trace.DCConnection.statusMsg:info]: AUTH: TraceDC- attempting authentication with domain controller \\REDDC1. Fri Feb 1 16:28:27 PST [red-str-napc2-p2: auth.trace.authenticateUser.loginRejected:info]: AUTH: Login attempt by user rejected by the domain controller with error 0xc0000199: STATUS_NOLOGON_WORKSTATION_TRUST_ACCOUNT. Fri Feb 1 16:28:28 PST [red-str-napc2-p2: auth.trace.authenticateUser.loginTraceIP:info]: AUTH: Login attempt by user red-inf-cm-p01$ of domain DOMAIN from client machine 1.1.1.1 (RED-INF-CM-P01). This reads like the problem is with AD rejecting the login, but when we point to another CIFS share on a real Windows box we don't get the same problem, so we don't think that's the case. We came across this KB article[1] which seems to disable the use of the computer account for authentication. However, this seems to send *no* authentication information at all (anonymous?) which of course is rejected as well. Help? We'll be reaching out to support as well. Thanks, Ray [1] https://kb.netapp.com/support/index?page=content&id=2013374

3 6

NT_STATUS_NO_MEMORY
by Ray Van Dolson 05 Feb '13

05 Feb '13

Anyone ever encountered this error when attempting to access a CIFS share on an ONTAP 8.0.x system? ("NT_STATUS_NO_MEMORY"). We encountered reports from users of intermittent failures to connect to CIFS shares. From Windows hosts, things would just time out. From Samba clients, the above NT_STATUS_NO_MEMORY error was the result. The "fix" ended up being to restart the CIFS service on the system, but wondering what the true root cause here might be. I'm unsure how accurate the error returned by the Samba client is -- in any case there doesn't appear to be a way to query memory utilization on the NetApp. Thanks, Ray

2 2

VSC 4.1 & Online alignments
by Scott Eno 05 Feb '13

05 Feb '13

Hi, We're trying to get some VM's aligned in our NFS-based VMware environment using the "online" alignment tool packaged in VSC. Since we are NFS, the tool needs a VMFS datastore to use to align the VM. When we run the tool and it moves the VM to the VMFS datastore, the VM no longer shows up as misaligned. However, if we move the VM back to an NFS datastore, it shows up as misaligned again. So, is this a one-way trip? NFS --> VMFS, but that VM can never go back?

4 10

RE: nasty virus
by Klise, Steve 04 Feb '13

04 Feb '13

To fast on the trigger.. I don't want to start up the thread about AV on or not on the filers (I run it on my filer), but we got hit with a nasty variant of this. Not sure why Trend didn't block it, but basically a bunch of my CIF folders were set to read only/hidden.. Not fun. There are some other reasons that need to be mitigated that I wont go into, but this is the virus. http://about-threats.trendmicro.com/Malware.aspx?id=47409&name=WORM_VOBFUS.… To unhide the share, we were mapped to the share location, and ran this.. It unhides the shares lick-ity split. >From an XP box.. Cmd "usebackq delims=#" %i in (`dir *. /ah /s /b`) do attrib -s -h "%i" Just share because I care..

2 1

instantaneous snap restore ?
by Avi Ben Emanuel (Wollman) 04 Feb '13

04 Feb '13

marketing continuously claim that snap restore is instantaneous. yet I have yet to succeeded doing so. "snap restore" of files to a location quite a while and is "copying the files" nothing close to instantaneous. i would be happy to understand what i am missing. the following is a example of a virtual disk that was restored, it took quite a while to "restore"=copy this 80gb file. snap restore -t file -s nightly.0 /vol/vmsata/vmfs1/lms-client/lms-clinet-flat.vmdk Avi -- ------------ Avi Ben Emanuel (Wollman) (אבי בן עמנואל (וולמן Systems Engineer | Unix Microsoft Vmware Jerusalem College of Technology מחלקת תקשוב בית הספר הגבוה לטכנולוגיה - מכון לב POB 16031 Jerusalem 91160 Israel הועד הלאומי 21 ירושלים Email: avi(a)jct.ac.il | Office: +972-2-6751036 | Mobile: +972-52-3809107 ------------

4 3

Re: Aggregate Disk Busy 100% with volume IOPS low
by Fletcher Cocquyt 27 Jan '13

27 Jan '13

Indeed, We are considering replacing our premium support with next day and using the savings to buy some professional services - we've heard other groups see better support ROI with this combination. On the wafl block reclamation - are you talking about options wafl.trunc.throttle.vol.max etc? we had to tune that back in 2010 7.3x days: http://www.vmadmin.info/2010/11/vfiler-migrate-netapp-lockup.html But not sure this is still a hidden option in 8.1.x? I read references to a tool called perfviewer - anyone still using that? thanks On Jan 26, 2013, at 1:45 PM, Isaiah <zoratu(a)gmail.com> wrote: > If I were you, I would purchase some incident-based support from Berkeley Communications ("Berkcom"). They're the only reseller of both used and new NetApp gear in the world. They know more about NetApp than NetApp. I've been a customer for nine years and they're the first number I call--because they're all experts. No escalation yadda yadda. Completely worth the modest fees to support gear not purchased through them. > > The last time your situation happened to me I ended up having to tune the wafl block reclamation aggressiveness. There were so many snapshots happening on the system with lots of gradual changes that the disk utilization was high just reclaiming blocks that used to be dirty. > -- > - Isaiah > > On Jan 26, 2013, at 10:15, Fletcher Cocquyt <fcocquyt(a)stanford.edu> wrote: > >> On Nick's advice I setup a job to log both wafltop and ps -c 1 once per minute - and we had a sustained sata0 disk busy from 5am-7am as reported by NMC. >> First question I have from wafltop show is - what is the first row (sata0::file i/o) reporting ? What could be the source of these 28907 non-volume specific Read IOs? >> >> Application MB Total MB Read(STD) MB Write(STD) Read IOs(STD) Write IOs(STD) >> ----------- -------- ------------ ------------- ------------- -------------- >> sata0::file i/o: 5860 5830 30 28907 0 >> sata0:backup:nfsv3: 608 0 608 31 0 >> >> I'm just starting to go through the data >> >> aggr status >> Aggr State Status Options >> sata0 online raid_dp, aggr nosnap=on, raidsize=12 >> 64-bit >> aggr2 online raid_dp, aggr nosnap=on, raidsize=19 >> 64-bit >> aggr1 online raid_dp, aggr root, nosnap=on, raidsize=14 >> 32-bit >> na04*> df -Ah >> Aggregate total used avail capacity >> aggr1 13TB 11TB 1431GB 89% >> aggr2 19TB 14TB 5305GB 74% >> sata0 27TB 19TB 8027GB 72% >> >> >> <sataIOPSJan26.jpeg> >> >> thanks >> >> >> On Jan 25, 2013, at 5:33 PM, Nicholas Bernstein <nick(a)nicholasbernstein.com> wrote: >> >>> Try doing a 'ps -c 1' or a wafltop show (double check the syntax) while you're getting the spike; those will probably help you narrow down the processes that are using your disks. Both are priv set advanced/diag commands. >>> >>> Nick >>> >> >> _______________________________________________ >> Toasters mailing list >> Toasters(a)teaparty.net >> http://www.teaparty.net/mailman/listinfo/toasters

1 0

Re: Aggregate Disk Busy 100% with volume IOPS low
by Fletcher Cocquyt 24 Jan '13

24 Jan '13

Hi Mike, this is a snap mirror SOURCE. The perfstats captured during the issue include wafl scan status - I'll mention it to the engineer when I show the disparity btw AGGR OPS and total volume OPS. Unfortunately its not easy to differentiate the source of IOPS with any existing tools in my experience - including perfstats and escalated Netapp support analysis. But the NMC AGGR vs volume stats need to be explained somehow. thanks On Jan 23, 2013, at 9:35 PM, Mike Nye <Mike.Nye(a)xpanse.com.au> wrote: > Hi Fletcher, > > Is this FAS a SnapMirror destination? This sounds very much like a volume deswizzle / CBR scan issue… > > You could try scheduling a “priv set advanced; wafl scan status” to run via RSH/SSH during the time of poor performance, to see what internal WAFL scans are taking place on the system. > > Thinks to look for are “Container Block Reclamation” and/or “Volume Deswizzle”. > > Kind regards, > Mike Nye > > > <image001.jpg> > <image002.jpg> > Mike Nye > Team Lead Systems Engineer > mike.nye(a)xpanse.com.au > www.xpanse.com.au > Mob: > Office: > Fax: > +61 407 772 465 > +61 8 9322 6767 > +61 8 9322 6077 > 18 Emerald Tce > West Perth > WA 6005 > <image003.jpg> > > > From: toasters-bounces(a)teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Fletcher Cocquyt > Sent: Thursday, 24 January 2013 1:28 PM > To: toasters(a)teaparty.net Lists > Subject: Aggregate Disk Busy 100% with volume IOPS low > > 3270 cluster, OnTAP 8.1-7mode > > We are investigating a SATA aggregate showing repeated 5am disk 100% busy spikes without its volumes showing any corresponding IOPS spike as reported by Netapp Management Console (NMC). > The 5am disk busy spikes correlate with very high latency on volumes on a different SAS aggregate. These volumes host VMs which then timeout, some needing reboots. > Today when I heard from Netapp support after reviewing my perfstat the engineer reported this is expected since NVRAM buffers are shared btw aggregates. > > But when I dig further into the NMC stats I see the SATA aggregate disk busy actually corresponds to a DROP in IOPS on the 3 volumes hosted on the SATA aggregate - almost like some internal aggregate operations are starving out the external volume ops. > I checked the snapshots (vol and aggr), snap mirror, dedup and none of the usual suspects were running. > > When I look at the NMC throughput graphs and switch on the legend - it shows a 5am READ blocks/sec spike corresponding perfectly to the disk busy. > > Where are these AGGR level READ operations coming from that are missing from the constituent volume IOPS, and in fact seem to be starving out volume level IO? > > I don't see much in the messages log, but will check the rest of the logs for internal type OPS > > thanks for any insight > > > > > >

2 1

Re: Aggregate Disk Busy 100% with volume IOPS low
by Fletcher Cocquyt 24 Jan '13

24 Jan '13

No NDMP in use for us - at a loss to explain this level of AGGR disk busy with no vol level IO Feels like an internal type operation hitting a bug thanks On Jan 23, 2013, at 10:21 PM, "Klise, Steve" <klises(a)sutterhealth.org> wrote: > A stab but what about ndmp jobs? > > From: Fletcher Cocquyt [mailto:fcocquyt@stanford.edu] > Sent: Wednesday, January 23, 2013 09:27 PM > To: toasters(a)teaparty.net Lists <toasters(a)teaparty.net> > Subject: Aggregate Disk Busy 100% with volume IOPS low > > 3270 cluster, OnTAP 8.1-7mode > > We are investigating a SATA aggregate showing repeated 5am disk 100% busy spikes without its volumes showing any corresponding IOPS spike as reported by Netapp Management Console (NMC). > The 5am disk busy spikes correlate with very high latency on volumes on a different SAS aggregate. These volumes host VMs which then timeout, some needing reboots. > Today when I heard from Netapp support after reviewing my perfstat the engineer reported this is expected since NVRAM buffers are shared btw aggregates. > > But when I dig further into the NMC stats I see the SATA aggregate disk busy actually corresponds to a DROP in IOPS on the 3 volumes hosted on the SATA aggregate - almost like some internal aggregate operations are starving out the external volume ops. > I checked the snapshots (vol and aggr), snap mirror, dedup and none of the usual suspects were running. > > When I look at the NMC throughput graphs and switch on the legend - it shows a 5am READ blocks/sec spike corresponding perfectly to the disk busy. > > Where are these AGGR level READ operations coming from that are missing from the constituent volume IOPS, and in fact seem to be starving out volume level IO? > > I don't see much in the messages log, but will check the rest of the logs for internal type OPS > > thanks for any insight > > > > > >

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters