toasters

toasters@lists.teaparty.net

3 participants
13534 discussions

Boot version info.
by Jeff Cleverley 24 May '13

24 May '13

Greetings, I'm replacing a 3140 with a 3270 stand-alone NearStore replacement. The current 3140 OS is 8.1.2P3 and the new head shipped with 8.1.2. Is that going to be close enough for the system to boot from the old system root disk, or does it have to be an exact match? If there is a way to modify the boot config from the command line instead of pulling the compact flash, I'd be OK with that also. Thanks, Jeff -- Jeff Cleverley Unix Systems Administrator 4380 Ziegler Road Fort Collins, Colorado 80525 970-288-4611

3 8

VM Virtual Disk Latency Versus NFS Latency on Filer
by Philbert Rupkins 22 May '13

22 May '13

Hello, This may be more of a VMWare question. From within Data Fabric Manager, I am seeing a spike in latency on a NetApp volume used as an NFS datastore. DFM shows a spike in NFS latency of 50 ms on the volume. However, a virtual machine's performance counters in vSphere show a spike in Virtual Disk latency upwards of 700ms for the same "latency event". Any idea what might account for this difference in latency reporting between the NetApp (DFM) and ESXi? I expected there to be some difference for obvious reasons but I didnt expect to see a difference of this magnitude. Thanks, Phil

3 5

Hot adding DS14MK4 trays to existing loop
by Fletcher Cocquyt 22 May '13

22 May '13

This is something I've never had reason to attempt until now. We have a 3270 cluster (8.1.2) Head 2 aggr1 consists of these two multipathed DS14MK4 trays (IDs 1,2): http://i.imgur.com/821iN35.jpg We want to hot add another DS14MK4 trays racked right above (IDs 3,4): http://imgur.com/FlMxpyz I've read the 2011 DS14MK4 guide and verified the loop speeds are all set consistently and IDs are all unique. (note: trays IDs are 1-4 from bottom to top) Can I simply: 1 - remove the top two pairs of fiber and transceivers (these go to head 1 (partner)) 2 - daisy chain up from those vacated ports to new trays (3,4) 3 - plug the fiber and tranceivers back in on the top tray? We would then grow (effectively double) this aggr1's capacity What are the risks ? (2011 guide mentions panic's are possible with loop misconfig) (we have 60 VMs running on head 2 on a separate loop and aggregate) Any other tips appreciated thanks

5 4

Oracle RAC on Netapp unstable under load
by Fletcher Cocquyt 20 May '13

20 May '13

Recently we have seen our Oracle RAC VMs reboot during netowkr and Netapp (Ontap 8.1.2 NFS) load correlated to our Friday 8pm full backup window start time. Initial strategies to prevent this include distributing RAC voting disks on different Netapp volumes, but I'd like to hear if others who have seen this have better solutions in practice? thanks, Fletcher

2 1

Delaying 64 bit aggregate upgrades
by Jeff Cleverley 17 May '13

17 May '13

Greetings, Is there a way to go from 7.3.5 to 8.1.2 without having all the aggregates start doing automatic conversions to 64 bit? Our current filers are getting hit pretty hard. I think the OS upgrade will help, but I'm afraid the aggregate upgrades will generate too much disk activity and slow things down for a day or two. We have ~9 aggregates per head that are 10T or larger. Most of these only have one volume in them. The shared ones have a few volumes with 40M+ inodes in use. If I can upgrade the aggregates selectively it will make it less noticeable to the users. Thanks, Jeff -- Jeff Cleverley Unix Systems Administrator 4380 Ziegler Road Fort Collins, Colorado 80525 970-288-4611

4 5

ONTAP 8.1.2P4 / Spike in Latency After Storage vMotion
by Philbert Rupkins 16 May '13

16 May '13

Hello, We were experiencing issues with large file deletes and spikes in latency from our Netapp filers. We run v3240's and FAS2240's. All are running 8.1.2GA or 8.1.1GA. The problem was only observed on those filers running 8.1.2GA. One operation that was observed to causing the problem consistently was Storage vMotions from an NFS datastore. In particular, the tail end of the Storage vMotion when the guest machine's files were deleted from the datastore seemed to trigger the problem. It looks like other people were experiencing the same issue. NetApp confirmed that a bug existed related to the deleting of large files. That issue was addressed in 8.1.2P3. We decided to deploy 8.1.2P4 as it included additional fixes related to deleting large files. Has anybody else deployed 8.1.2P4 yet? We just completed our deploy of 8.1.2P4 and it seems to have resolved the issue with large file deletes. We've tested a few Storage vMotions and removals of VM's with no issues thus far. If you are using 8.1.2P4, have you observed other issues? Thanks, Phil

3 4

FAS backup question
by Mike Gossett 15 May '13

15 May '13

Greetings: I have a general question in regards to backup solutions for NetApp filers. We have a client that is in the process of purchasing a FAS-2240-2 HA, and will be purchasing a FAS2220 for a DR site later this year. We are getting the complete license bundle to have SnapManager backups for VMWare, Exchange, etc to local disk, and licensing SnapMirror on the FAS2220 for replication of current data. It is our intent to implement some slower SATA disks to maintain longer retention windows. However, this environment is very interested in additional backup measures beyond that. Historically we have been using BackupExec, using agents on physical servers and VMs, to backup to both a NAS device and LTO autoloader. I am a little confused on how we can effectively accomplish an efficient backup of these filers outside of the netapp ecosystem. I have seen evidence of other users with older or larger models using direct-attach SCSI tape autoloaders (or FC autoloaders) with NDMP. We can't implement the SCSI cards in the 2240, and we don't wish to have the budgetary hit of implementing FC solely for tape connections. We were prefer to avoid tape altogether, if at all possible, as these filers will be in collocation facilities that we don't otherwise physically visit. Some other background: most\all of the data on these 2240s will be VMWare VMs stored on NFS (thin) datastores. There may be one or two iSCSI connections from VMWare for SQL and Exchange for SME reasons. The environment is a MS shop, with Sharepoint\SQL being the critical business application. Does anyone have any recommendations about other ways to backup our data? We hope to rely on the NetApp replication and SnapManager\SnapRestore\SnapVault for most of our normal recovery needs, however require the safety net of a weekly or monthly backup to some other media. We also hope to avoid something that is too difficult to manage, and were hoping for a commercial solution. Backup Exec with appropriate agents and a local NAS device or something along those lines is what I was thinking. Does anyone have any experience with tape devices connected to a media server backing up NetApp filers? Can NDMP stream the content to the NDMP backup server with a locally attached autoloader? How about some other sort of disk target (e.g. cheap NAS appliance connected to the backup server via cifs \ iSCSI) I've reached out to my SE, who indicated that most organizations are either using FC\SAS\SCSI autoloaders with NDMP or are relying on the built-in NetApp SnapVault solution. I would welcome any guidance or experience that anyone can share. Thanks, Mike Gossett

5 7

Re: RAID not reconstructing on FAS3050c
by Chris Daniel 10 May '13

10 May '13

Doug, Thank you so much for your detailed reply! I have deleted the core files, removed the bad disks (now dblade 'aggr status -f' is empty), and cleared the shelf fault. But nothing else. Did some more poking around, and yes I agree, the available spares seem like they should get used ... the list of spares says "Spare disks for block or zoned checksum traditional volumes or aggregates" and my degraded aggregate says it is "block checksum". All the disks are the same block sizes, etc. Now, I couldn't figure out how to turn off the "ignore inconsistent" flag. Here's what I tried: in the cluster shell (what you get when you log in as admin(a)ip.ad.dr.ess), I ran 'set -privilege advanced' (as that seems to be what your prompt indicated), and tried 'aggr modify ...' but 'aggr' was not available. Then I tried 'storage aggregate modify ...' but there was no -ignore-inconsistent switch available (just a few: -state, -raidtype, -maxraidsize). Last, I tried running 'aggr modify ...' under the node shell (what you get when you run 'dblade' from the cluster shell), and there was no 'modify' subcommand available -- just the following: media_scrub, read_fsid, show_space, wafliron, raidlabel, scrub, status. Also, one last funny thing: looking at 'aggr status -r' in dblade, the RAID group with the failed disk seems to be stuck at "recomputing parity 0% completed". If you have any other input you are willing to provide free, I would appreciate it loads! If you'd like to take this off list let me know, more than willing to pay your fee. Thank you! On Thu, May 9, 2013 at 6:48 PM, Doug Siggins <DSiggins(a)ma.maileig.com>wrote: > 1. I hope you have a backup of this data! please make sure you at least > have a mirror (on a different aggregate)! > > Before anything else get those failed disks out of there and save them for > a short bit, and yes you will need to run a WAFL check, although it should > still rebuild while inconsistent. > Delete some cores off of the head: > ngsh*> coredump delete ? this might take a few tries, as it looks like its > trying to save the core files :) > > > or you could always rm them from /mroot/kcores > I find that much quicker. > > Clear the shelf fault: > May 5 07:00:00 [statd]: monitor.shelf.fault: Fault reported on disk > storage shelf attached to channel 0b. Please check fans, power, and > temperature. > > > > Broken disks > > RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) > Phys (MB/blks) > --------- ------ ------------- ---- ---- ---- ----- -------------- > -------------- > failed 0a.23 0a 1 7 FC:A - ATA 7200 > 423111/866531584 423889/868126304 > failed 0b.41 0b 2 9 FC:A - ATA 7200 > 423111/866531584 423889/868126304 > failed 0b.44 0b 2 12 FC:A - ATA 7200 > 423111/866531584 423889/868126304 > failed 0c.53 0c 3 5 FC:B - ATA 7200 > 423111/866531584 423889/868126304 > failed 0c.57 0c 3 9 FC:B - ATA 7200 > 423111/866531584 423889/868126304 > failed 0c.60 0c 3 12 FC:B - ATA 7200 > 423111/866531584 423889/868126304 > > you can light up the failed disks like this: > storage disk setled -disk bosback01:0a.98 -action on > > -action off to turn them off :) > > > The message below does explain the issue with it not rebuilding the 520 > bps thing, though that is odd. they look iike the same darned disks. > Whomever administered this machine over rode the inconsistent flag (which > is a horrible idea). So essentially you have a volume online while the fs > is inconsistent, and can run into issues where it panics. Turn that flag > off. > > bosback01::*> aggr modify -aggregate bosback01_aggr1 -ignore-inconsistent > off > > I think you can then offline the aggregate and run the wafl iron > (dbladecli aggr wafliron) while the system is online. Of course its always > safer to boot into maint mode (ctrl c while booting) and then run the wafl > check. It's been so long since I've done that and I cannot login to maint > mode to try it :) > > https://kb.netapp.com/support/index?page=content&id=3013616 > > I'd need to research the 520 bps thing further to figure this out, though > try the options raid.disktype raid.whatever under the dblade. > > Of course if you run into deeper issues and you need some help I can be > had for a somewhat cheap consulting fee :) > > You have lots of issues here, and are a little precarious with data > integrity. Again I hope you have a backup of the data. Have fun. > > > > ------------------------------ > *From:* toasters-bounces(a)teaparty.net [toasters-bounces(a)teaparty.net] on > behalf of Chris Daniel [cjdaniel(a)gmail.com] > *Sent:* Thursday, May 09, 2013 8:44 PM > *To:* toasters(a)teaparty.net > > *Subject:* Re: RAID not reconstructing on FAS3050c > > Thanks all! So far I have been able to get into the node shell and > determine that the aggregate is marked as "wafl inconsistent". > > The disks are all the same size and model (SATA connected via FC, NetApp > X267_MGRIZ500SSX), except for one, which is model X267_HKURO500SSX -- it > appears to be of the same size as the rest, however, and is currently a > good data disk. > > Around the time of the failure, this message appears in the messages.ems > log: > [config_thread]: raid.rg.recons.cantStart: The reconstruction cannot start > in RAID group /engdata1/plex0/rg2: No 520 bps disk of required size > available in spare pool > > That might explain why it's not reconstructing. The full messages.ems > log is here: http://pastebin.com/b9Xgj628 Also here is the output of > 'aggr status -r': http://pastebin.com/MmHFNbiz > > I'm guessing that what I need to do is run WAFL_check? Anything else I > should look at before I run off and do this? > > > > On Thu, May 9, 2013 at 2:03 PM, Doug Siggins <DSiggins(a)ma.maileig.com>wrote: > >> This is GX, he doesn't have the option to view the messages file >> without logging into the CLI: >> >> root@mgmt1 ip, cd /mroot/log >> there you can cat messages.log, and to view the messages.ems use >> ems_logviewer command: >> >> Last login: Wed May 8 20:01:17 2013 from 10.1.91.10 >> bosback01# cd /mroot/log >> bosback01# ems_logviewer messages.ems >> >> That will give some more information. Although I have never had this >> issue I have had GX from 10.0 to 10.0.4 In fact if you are still on GX, I >> have no idea why you would stick with 10.0.1, its horribly unstable! >> >> you can also get more information via storage disk show >> toast1a:0d.(number) >> >> One thing, those look like 450/500G disks. >> >> Are you using the same type of disk in the aggregate? (SATA/FC?) >> >> You may need to login to the dbladecli and allow the aggregate to use a >> different sized disk to rebuild onto (options disk or raid or something) >> >> Or the other thing I can think of is that you put an unsupported disk >> into that shelf. I know when I put in the 450G 15k FC I had to load up a >> new "unsupported" disk_qual package to see all of the space and use them. >> Which given Netapps congenial attitude towards screwing us GX users with >> FAS3050s didn't bother me the most to hack to get working :) >> >> >> I have been through a lot with GX/C-mode, and know quite a bit. I am >> looking for others who know things like zsmcli, and vldbtest :) >> >> >> >> ------------------------------ >> *From:* toasters-bounces(a)teaparty.net [toasters-bounces(a)teaparty.net] on >> behalf of tmac [tmacmd(a)gmail.com] >> *Sent:* Thursday, May 09, 2013 4:45 PM >> *To:* Chris Daniel >> *Cc:* Toasters >> *Subject:* Re: RAID not reconstructing on FAS3050c >> >> Try looking at it from the node perspective: >> (i forget the syntax offhand as it is slightly different that Ontap 8.1+ >> Cluster Mode) >> >> After you get into the node shell, do a disk show -n (make sure disks >> are properly assigned) >> >> try a aggr status -r >> >> Maybe the spares are zeroing before they are being used...that may be >> what pending is. >> >> Have you checked the event log? what about the messages file? >> >> --tmac >> >> *Tim McCarthy* >> *Principal Consultant* >> >> >> >> >> Clustered ONTAP >> Clustered ONTAP >> NCDA ID: XK7R3GEKC1QQ2LVD RHCE5 805007643429572 NCSIE ID: >> C14QPHE21FR4YWD4 >> Expires: 08 November 2014 Expires w/release of RHEL7 >> Expires: 08 November 2014 >> >> >> >> On Thu, May 9, 2013 at 4:32 PM, Chris Daniel <cjdaniel(a)gmail.com> wrote: >> >>> Hello, >>> >>> We've got a couple of FAS3050c filers running 4 DS14mk2 shelves full of >>> disks (each filer is connected to two disk shelves). They've mostly been >>> trouble-free, but this week it seems a disk failed, and our main storage >>> aggregate went into "degraded" mode. For some reason, despite spare disks >>> being available, it's not reconstructing as I would think it should. >>> >>> The software running on these filers is Data ONTAP GX 10.0.1P2 -- from >>> previous discussions with the community, I've learned that GX has a whole >>> different set of commands, so many of the Google-able resources I've found >>> aren't relevant. Adding to that difficulty, we don't have a support >>> contract on these filers (but they are properly licensed and whathaveyou). >>> >>> Here is the output of 'storage aggregate show -aggregate engdata1' >>> (that is the degraded aggregate): >>> >>> toast1a::> storage aggregate show -aggregate engdata1 >>> >>> Aggregate: engdata1 >>> Size (MB): 0 >>> Used Size (MB): 0 >>> Used Percentage: - >>> Available Size (MB): 0 >>> State: restricted >>> Nodes: toast1a >>> Number Of Disks: 37 >>> Disks: toast1a:0a.16, toast1a:0b.32, toast1a:0c.48, >>> toast1a:0a.17, toast1a:0b.33, toast1a:0c.49, >>> toast1a:0a.18, toast1a:0b.34, toast1a:0c.50, >>> toast1a:0a.19, toast1a:0b.35, toast1a:0c.51, >>> toast1a:0a.20, toast1a:0d.64, toast1a:0b.37, >>> toast1a:0a.21, toast1a:0c.52, toast1a:0b.38, >>> toast1a:0a.22, toast1a:0c.61, toast1a:0b.39, >>> toast1a:0d.69, toast1a:0c.54, toast1a:0b.40, >>> toast1a:0a.24, toast1a:0c.55, toast1a:0d.65, >>> toast1a:0a.25, toast1a:0a.26, toast1a:0b.42, >>> toast1a:0c.59, toast1a:0a.27, toast1a:0b.43, >>> toast1a:0a.28, toast1a:0b.45, toast1a:0d.68, >>> toast1a:0d.71 >>> Number Of Volumes: 0 >>> Plexes: /engdata1/plex0(online) >>> RAID Groups: /engdata1/plex0/rg0, /engdata1/plex0/rg1, >>> /engdata1/plex0/rg2 >>> Raid Type: raid_dp >>> Max RAID Size: 14 >>> RAID Status: raid_dp,degraded >>> Checksum Enabled: true >>> Checksum Status: active >>> Checksum Style: block >>> Inconsistent: true >>> Volume Types: flex >>> >>> There are spare disks available now, but there were not when the >>> failure occurred. I moved two spare disks to the right filer after the >>> failure, thinking that would cause the aggregate to start reconstructing. >>> Here is the output of 'storage disk show -state spare': >>> >>> toast1a::> storage disk show -state spare >>> Disk UsedSize(MB) Shelf Bay State RAID Type Aggregate >>> Owner >>> ---------------- ------------ ----- --- --------- ---------- --------- >>> -------- >>> toast1a:0d.72 423090 4 8 spare pending - >>> toast1a >>> toast1a:0d.73 423090 4 9 spare pending - >>> toast1a >>> toast1b:0d.74 423090 4 10 spare pending - >>> toast1b >>> toast1b:0d.75 423090 4 11 spare pending - >>> toast1b >>> toast1b:0d.76 423090 4 12 spare pending - >>> toast1b >>> toast1b:0d.77 423090 4 13 spare pending - >>> toast1b >>> 6 entries were displayed. >>> >>> Can anyone provide insight on this problem? Why is the aggregate not >>> reconstructing when there are spares available? NetApp stuff is not my >>> specialty, but I'm the one who gets to deal with it, and I am pretty >>> stumped. Thank you in advance! >>> >>> -- >>> Chris Daniel >>> >>> _______________________________________________ >>> Toasters mailing list >>> Toasters(a)teaparty.net >>> http://www.teaparty.net/mailman/listinfo/toasters >>> >>> >> > > > -- > Chris Daniel > -- Chris Daniel

1 0

Re: RAID not reconstructing on FAS3050c
by Chris Daniel 10 May '13

10 May '13

Thanks all! So far I have been able to get into the node shell and determine that the aggregate is marked as "wafl inconsistent". The disks are all the same size and model (SATA connected via FC, NetApp X267_MGRIZ500SSX), except for one, which is model X267_HKURO500SSX -- it appears to be of the same size as the rest, however, and is currently a good data disk. Around the time of the failure, this message appears in the messages.ems log: [config_thread]: raid.rg.recons.cantStart: The reconstruction cannot start in RAID group /engdata1/plex0/rg2: No 520 bps disk of required size available in spare pool That might explain why it's not reconstructing. The full messages.ems log is here: http://pastebin.com/b9Xgj628 Also here is the output of 'aggr status -r': http://pastebin.com/MmHFNbiz I'm guessing that what I need to do is run WAFL_check? Anything else I should look at before I run off and do this? On Thu, May 9, 2013 at 2:03 PM, Doug Siggins <DSiggins(a)ma.maileig.com>wrote: > This is GX, he doesn't have the option to view the messages file without > logging into the CLI: > > root@mgmt1 ip, cd /mroot/log > there you can cat messages.log, and to view the messages.ems use > ems_logviewer command: > > Last login: Wed May 8 20:01:17 2013 from 10.1.91.10 > bosback01# cd /mroot/log > bosback01# ems_logviewer messages.ems > > That will give some more information. Although I have never had this issue > I have had GX from 10.0 to 10.0.4 In fact if you are still on GX, I have no > idea why you would stick with 10.0.1, its horribly unstable! > > you can also get more information via storage disk show toast1a:0d.(number) > > One thing, those look like 450/500G disks. > > Are you using the same type of disk in the aggregate? (SATA/FC?) > > You may need to login to the dbladecli and allow the aggregate to use a > different sized disk to rebuild onto (options disk or raid or something) > > Or the other thing I can think of is that you put an unsupported disk into > that shelf. I know when I put in the 450G 15k FC I had to load up a new > "unsupported" disk_qual package to see all of the space and use them. Which > given Netapps congenial attitude towards screwing us GX users with FAS3050s > didn't bother me the most to hack to get working :) > > > I have been through a lot with GX/C-mode, and know quite a bit. I am > looking for others who know things like zsmcli, and vldbtest :) > > > > ------------------------------ > *From:* toasters-bounces(a)teaparty.net [toasters-bounces(a)teaparty.net] on > behalf of tmac [tmacmd(a)gmail.com] > *Sent:* Thursday, May 09, 2013 4:45 PM > *To:* Chris Daniel > *Cc:* Toasters > *Subject:* Re: RAID not reconstructing on FAS3050c > > Try looking at it from the node perspective: > (i forget the syntax offhand as it is slightly different that Ontap 8.1+ > Cluster Mode) > > After you get into the node shell, do a disk show -n (make sure disks > are properly assigned) > > try a aggr status -r > > Maybe the spares are zeroing before they are being used...that may be > what pending is. > > Have you checked the event log? what about the messages file? > > --tmac > > *Tim McCarthy* > *Principal Consultant* > > > > > Clustered ONTAP > Clustered ONTAP > NCDA ID: XK7R3GEKC1QQ2LVD RHCE5 805007643429572 NCSIE ID: > C14QPHE21FR4YWD4 > Expires: 08 November 2014 Expires w/release of RHEL7 > Expires: 08 November 2014 > > > > On Thu, May 9, 2013 at 4:32 PM, Chris Daniel <cjdaniel(a)gmail.com> wrote: > >> Hello, >> >> We've got a couple of FAS3050c filers running 4 DS14mk2 shelves full of >> disks (each filer is connected to two disk shelves). They've mostly been >> trouble-free, but this week it seems a disk failed, and our main storage >> aggregate went into "degraded" mode. For some reason, despite spare disks >> being available, it's not reconstructing as I would think it should. >> >> The software running on these filers is Data ONTAP GX 10.0.1P2 -- from >> previous discussions with the community, I've learned that GX has a whole >> different set of commands, so many of the Google-able resources I've found >> aren't relevant. Adding to that difficulty, we don't have a support >> contract on these filers (but they are properly licensed and whathaveyou). >> >> Here is the output of 'storage aggregate show -aggregate engdata1' >> (that is the degraded aggregate): >> >> toast1a::> storage aggregate show -aggregate engdata1 >> >> Aggregate: engdata1 >> Size (MB): 0 >> Used Size (MB): 0 >> Used Percentage: - >> Available Size (MB): 0 >> State: restricted >> Nodes: toast1a >> Number Of Disks: 37 >> Disks: toast1a:0a.16, toast1a:0b.32, toast1a:0c.48, >> toast1a:0a.17, toast1a:0b.33, toast1a:0c.49, >> toast1a:0a.18, toast1a:0b.34, toast1a:0c.50, >> toast1a:0a.19, toast1a:0b.35, toast1a:0c.51, >> toast1a:0a.20, toast1a:0d.64, toast1a:0b.37, >> toast1a:0a.21, toast1a:0c.52, toast1a:0b.38, >> toast1a:0a.22, toast1a:0c.61, toast1a:0b.39, >> toast1a:0d.69, toast1a:0c.54, toast1a:0b.40, >> toast1a:0a.24, toast1a:0c.55, toast1a:0d.65, >> toast1a:0a.25, toast1a:0a.26, toast1a:0b.42, >> toast1a:0c.59, toast1a:0a.27, toast1a:0b.43, >> toast1a:0a.28, toast1a:0b.45, toast1a:0d.68, >> toast1a:0d.71 >> Number Of Volumes: 0 >> Plexes: /engdata1/plex0(online) >> RAID Groups: /engdata1/plex0/rg0, /engdata1/plex0/rg1, >> /engdata1/plex0/rg2 >> Raid Type: raid_dp >> Max RAID Size: 14 >> RAID Status: raid_dp,degraded >> Checksum Enabled: true >> Checksum Status: active >> Checksum Style: block >> Inconsistent: true >> Volume Types: flex >> >> There are spare disks available now, but there were not when the >> failure occurred. I moved two spare disks to the right filer after the >> failure, thinking that would cause the aggregate to start reconstructing. >> Here is the output of 'storage disk show -state spare': >> >> toast1a::> storage disk show -state spare >> Disk UsedSize(MB) Shelf Bay State RAID Type Aggregate >> Owner >> ---------------- ------------ ----- --- --------- ---------- --------- >> -------- >> toast1a:0d.72 423090 4 8 spare pending - >> toast1a >> toast1a:0d.73 423090 4 9 spare pending - >> toast1a >> toast1b:0d.74 423090 4 10 spare pending - >> toast1b >> toast1b:0d.75 423090 4 11 spare pending - >> toast1b >> toast1b:0d.76 423090 4 12 spare pending - >> toast1b >> toast1b:0d.77 423090 4 13 spare pending - >> toast1b >> 6 entries were displayed. >> >> Can anyone provide insight on this problem? Why is the aggregate not >> reconstructing when there are spares available? NetApp stuff is not my >> specialty, but I'm the one who gets to deal with it, and I am pretty >> stumped. Thank you in advance! >> >> -- >> Chris Daniel >> >> _______________________________________________ >> Toasters mailing list >> Toasters(a)teaparty.net >> http://www.teaparty.net/mailman/listinfo/toasters >> >> > -- Chris Daniel

2 1

RAID not reconstructing on FAS3050c
by Chris Daniel 09 May '13

09 May '13

Hello, We've got a couple of FAS3050c filers running 4 DS14mk2 shelves full of disks (each filer is connected to two disk shelves). They've mostly been trouble-free, but this week it seems a disk failed, and our main storage aggregate went into "degraded" mode. For some reason, despite spare disks being available, it's not reconstructing as I would think it should. The software running on these filers is Data ONTAP GX 10.0.1P2 -- from previous discussions with the community, I've learned that GX has a whole different set of commands, so many of the Google-able resources I've found aren't relevant. Adding to that difficulty, we don't have a support contract on these filers (but they are properly licensed and whathaveyou). Here is the output of 'storage aggregate show -aggregate engdata1' (that is the degraded aggregate): toast1a::> storage aggregate show -aggregate engdata1 Aggregate: engdata1 Size (MB): 0 Used Size (MB): 0 Used Percentage: - Available Size (MB): 0 State: restricted Nodes: toast1a Number Of Disks: 37 Disks: toast1a:0a.16, toast1a:0b.32, toast1a:0c.48, toast1a:0a.17, toast1a:0b.33, toast1a:0c.49, toast1a:0a.18, toast1a:0b.34, toast1a:0c.50, toast1a:0a.19, toast1a:0b.35, toast1a:0c.51, toast1a:0a.20, toast1a:0d.64, toast1a:0b.37, toast1a:0a.21, toast1a:0c.52, toast1a:0b.38, toast1a:0a.22, toast1a:0c.61, toast1a:0b.39, toast1a:0d.69, toast1a:0c.54, toast1a:0b.40, toast1a:0a.24, toast1a:0c.55, toast1a:0d.65, toast1a:0a.25, toast1a:0a.26, toast1a:0b.42, toast1a:0c.59, toast1a:0a.27, toast1a:0b.43, toast1a:0a.28, toast1a:0b.45, toast1a:0d.68, toast1a:0d.71 Number Of Volumes: 0 Plexes: /engdata1/plex0(online) RAID Groups: /engdata1/plex0/rg0, /engdata1/plex0/rg1, /engdata1/plex0/rg2 Raid Type: raid_dp Max RAID Size: 14 RAID Status: raid_dp,degraded Checksum Enabled: true Checksum Status: active Checksum Style: block Inconsistent: true Volume Types: flex There are spare disks available now, but there were not when the failure occurred. I moved two spare disks to the right filer after the failure, thinking that would cause the aggregate to start reconstructing. Here is the output of 'storage disk show -state spare': toast1a::> storage disk show -state spare Disk UsedSize(MB) Shelf Bay State RAID Type Aggregate Owner ---------------- ------------ ----- --- --------- ---------- --------- -------- toast1a:0d.72 423090 4 8 spare pending - toast1a toast1a:0d.73 423090 4 9 spare pending - toast1a toast1b:0d.74 423090 4 10 spare pending - toast1b toast1b:0d.75 423090 4 11 spare pending - toast1b toast1b:0d.76 423090 4 12 spare pending - toast1b toast1b:0d.77 423090 4 13 spare pending - toast1b 6 entries were displayed. Can anyone provide insight on this problem? Why is the aggregate not reconstructing when there are spares available? NetApp stuff is not my specialty, but I'm the one who gets to deal with it, and I am pretty stumped. Thank you in advance! -- Chris Daniel

3 2

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters