toasters

toasters@lists.teaparty.net

2 participants
13517 discussions

Re: Weird mass delete of files in a volume with ONTAP ASYNC-Delete
by John Stoffel 22 Jul '25

22 Jul '25

>>>>> "Paul" == Paul Letta via toasters <toasters(a)lists.teaparty.net> writes: > From: Paul Letta <letta(a)jlab.org> > To: toasters <toasters(a)teaparty.net> > Date: Tue, 15 Jul 2025 20:57:00 +0000 > Subject: Weird mass delete of files in a volume with ONTAP ASYNC-Delete > Hi Toasters. > I’m running 9.14.1P13. > I used the ONTAP ASYNC-Delete feature from the GUI (storage – > Volumes – click on a volume – Click on File system – click on > Explorer) to delete a few top level folders in a volume. That > worked fine. I saw the background job running and they finished > fine. > I needed to delete much more. So I clicked on the trash can for one > of the folders, clicked ok to start the delete. Why not do the delete from a CIFS share instead? > Then I clicked elsewhere in the GUI and clicked back to File-System, > Explorer – and clicked on the trash can for a different folder. Hmm... I wonder if you left something selected elsewhere, or even up a level when you started one of the deletes? > I did this maybe 6 or 7 times. When I check the jobs in ONTAP, I > saw them all running. > But then as I waited, I saw a lot more space being used by snapshots > (via the df command) than the 6 or 7 folders I wanted to delete. > At this point, only 1 ASYNC-Delete job was still running, so I canceled it. > When I look at the live file system.. I see almost all folders (not > just the 6 or 7 I clicked) have massive files deleted. Luckily for > me, this volume was just a filesystem that barely changes.. so I > could snapshot restore from a few hours before, and all was well. Whew! Love snapshots. > Now I’m scared to death of that file-explore page in the ONTAP GUI. > I am the only administrator, so no one else could have been logged > on to ONTAP. The pattern of deleted files makes a delete coming > from a CIFS client very unlikely. There is no NFS export to this > volume. I've never used it, but I'd probably not ever use it unless I had a problem doing the deletes from a client instead. But I'm much more in tune with NFS than CIFS stuff. > Maybe the file explorer delete button was not intended for multiple, > simultaneous runs? Do you have the space to clone the volume to a test copy and play around and see if you can recreate the problem? > I don’t know.. I know I will avoid page like a plague forever now. > Any thoughts ? Do all filesystem manipulations from the clients if at all possible in the future. John

1 0

Weird mass delete of files in a volume with ONTAP ASYNC-Delete
by Paul Letta 15 Jul '25

15 Jul '25

Hi Toasters. I'm running 9.14.1P13. I used the ONTAP ASYNC-Delete feature from the GUI (storage - Volumes - click on a volume - Click on File system - click on Explorer) to delete a few top level folders in a volume. That worked fine. I saw the background job running and they finished fine. I needed to delete much more. So I clicked on the trash can for one of the folders, clicked ok to start the delete. Then I clicked elsewhere in the GUI and clicked back to File-System, Explorer - and clicked on the trash can for a different folder. I did this maybe 6 or 7 times. When I check the jobs in ONTAP, I saw them all running. But then as I waited, I saw a lot more space being used by snapshots (via the df command) than the 6 or 7 folders I wanted to delete. At this point, only 1 ASYNC-Delete job was still running, so I canceled it. When I look at the live file system.. I see almost all folders (not just the 6 or 7 I clicked) have massive files deleted. Luckily for me, this volume was just a filesystem that barely changes.. so I could snapshot restore from a few hours before, and all was well. Now I'm scared to death of that file-explore page in the ONTAP GUI. I am the only administrator, so no one else could have been logged on to ONTAP. The pattern of deleted files makes a delete coming from a CIFS client very unlikely. There is no NFS export to this volume. Maybe the file explorer delete button was not intended for multiple, simultaneous runs? I don't know.. I know I will avoid page like a plague forever now. Any thoughts ? Thanks!

1 0

Large Amount of Aggregate Metadata
by Stephen Stocke 01 May '25

01 May '25

Hi All We have a couple AFF-A250 cluster running 9.15.1P7 which host NFS data stores for vSphere. Most of the volumes on this particular cluster are SnapMirror destination volumes at our secondary data center. When running 'df -A' to check the data distribution on the cluster, I noticed that one aggregate was showing 100% utilisation. Running 'storage aggregate show-space' indicates that 72% of the space (32TB!) is being consumed by aggregate metadata. What might cause so much metadata in the aggregate? Are there any commands to see which volume the metadata is associated with? Also, when logging on to the web console, there was no indication of a problem or unhealthy node so I'm not sure if the aggregate was actually full. I ran a vol move to the partner data aggregate successfully so I now have a bit of free space. (There is also a question as to why our monitoring system wasn't reporting any issues... but if system manager doesn't report an issue...?) Thanks, Steve

1 0

Igfrp IP Distribution Function
by Stephen Stocke 05 Feb '25

05 Feb '25

Greetings fellow toaster admins I hope someone can shed some light on the IP distribution function for lacp ifgrps. We have a four port, multi mode lacp ifgrp, a0a, using interfaces e2a-e2d. We observe SnapMirror traffic egressing port e2c. The XOR of the last two bits of the source and destination IPs are either x0 or x3 so I am expecting traffic to egress either the first or fourth port in the ifgrp. Assuming e2a is port 0, e2b is port 1, etc., I would expect traffic to egress either e2a (0x0) or e2d (0x3). What am I missing? I can’t find any details about the actual hashing function for IP distribution or port member indexing in an ifgrp to confirm my assumptions. Both clusters are a single A250 HA pair running 9.15.1. I’m typing this with my thumbs so apologies for any typos or insufficient detail. Best wishes Stephen

3 4

Unable to schedule service processor update
by Alexander Griesser 14 Jan '25

14 Jan '25

Hey folks, I got a bunch of A220s here which I want to update to the latest SP firmware: CLUSTER::*> system service-processor show IP Firmware Node Type Status Configured Version IP Address ------------- ---- ----------- ------------ --------- ------------------------- Node1 BMC online true 11.9P1 127.0.0.1 Node2 BMC online true 11.9P1 127.0.0.2 2 entries were displayed. But everytime I try to install the update from the Ontap prompt, I get the following message: CLUSTER::*> system service-processor image update -node * -package SP_FW_308-04195_11.11.tar.gz Note: Firmware update will need to reboot the SP on completion. If your console connection is through the SP, it will be disconnected Do you want to proceed with the firmware update ? {y|n}: y Error: command failed on node "Node1": Unable to schedule Service Processor update. The only KB article I found for this issue is this one: https://kb.netapp.com/on-prem/ontap/OHW/OHW-KBs/BMC_upgrade_fails_with_Erro… I can verify that the file is not corrupted (compared the MD5 sums) and I tried to reboot the SP before the update, but all to no avail. The only way I can get the BMC firmware update installed is by using "bmc update -f" directly on the bmc shell, but that's harder to automate. Has anyone ever stumbled upon that and has an idea? SP Auto Updates are enabled, but they're not starting either - waited for a day for them to do their job. Thanks, Alexander Griesser Head of Systems Operations ANEXIA Internetdienstleistungs GmbH E-Mail: AGriesser(a)anexia.com<mailto:AGriesser@anexia.com> Web: https://www.anexia.com<https://www.anexia.com/> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

3 4

S3 Buckets hosted on NetApp and Backups
by John Adams 30 Oct '24

30 Oct '24

Hi Everyone. I was wondering what is being done to backup S3 buckets hosted on NetApp. Can anyone share what they are doing, aside from the normal NetApp snapshot and mirroring things. Thanks !

7 15

iscsi setup walk through
by John Adams 22 Aug '24

22 Aug '24

Hello. Does anyone out there have a walk through of setting up iscsi targets in OnTap 9.11? Thanks.

1 0

Administrivia
by Tom Yates 18 Mar '24

18 Mar '24

This mailing list, like a number of others, is hosted on my colocated box, using mailman2. The box is running CentOS 7, which goes end-of-life fairly soon, and I plan to rebuild it running Debian 12 some time in the next month or two (more solid notice will be given nearer the time!). There have been no posts to this list since Oct 2023, so although we still have a goodly number of subscribers, I wonder whether, after 26 years, the list has run its course. I have enough other lists that I'll still have to put mailman3 on the new box, but migrating a large list with a big archive from mailman2 to mailman3 is still quite an undertaking. If it turned out that it was pointless, it would be a large enough job that not having to do it would be a noticeable time-saver. Let me clarify that I'm not looking to offload the list, or to move it again. If it's still useful, I'll do the legwork and migrate it; if it's not, then let's just turn it off. So I seek guidance. Let's not have the discussion on-list, but if you have any thoughts about whether this list is still useful, please let me know privately. Both positive and negative thoughts are valuable, so if you have any strong feelings either way, I'd like to hear them. Thanks! -- Tom Yates - https://www.teaparty.net

1 1

Re: Question about flash pool maximum SSD size and local tiering
by Michael Bergman 18 Oct '23

18 Oct '23

On 2023-10-17 14:54, Florian Schmid via Toasters wrote: > Hi Johan, > thank you very much for your help. > > No, we don't have the disks yet for which the flash-pool should be used. > Not all SSDs will be used for flash-pool, only some for cache and the rest > for fast SSD storage. So you're thinking to have several different "physical tiers" with different characteristics (performance, inherent latency) for different workloads -- in the same HA-pair? Several different Aggrs with differing performance and behaviour in the same node, FAS8300? (it's a fairly powerful machine so it can do this adequately in many smaller workload cases). Or do you mean in different 8300 nodes in a X-node cluster (what's X?) This idea is much harder to make successful than you probably think. It requires you to know very much about your workloads, your applications, what they do so that you can place the correct data in the right place and you have to have the ability to do this over time as data volumes grow. Assuming they do... It's very hard indeed to automate so you need people who can baby watch this continuously and move data around. Yes that's mostly non-disruptive, but it's still quite a lot of work. It also pretty much assumes for it to be successful in the longer run that your applications do not change their workload patterns and/or pressure more than very slowly. Is this the case? All in all FabricPool is much much more automatic. It just does the job itself, pretty much w/o fuss once you've tuned it a bit w.r.t. cool down period(s) and things. It "just works". You do need an S3 target system, but as has already been pointed out it can be ONTAP with NL-SAS drives, if you already have a bunch of these lying about you can repurpose those and instead use new Cx00 (or Ax00) nodes in the "front end". The challenge with FabricPool is the network: the connection between the front end and the S3 back end needs to be very good and solid. You have to understand it fully and know every detail how it's built so you know you can trust it's capacity and latency; traffic can be quite bursty. I'm not very positive to your idea here I'm afraid: "Not all SSDs will be used for flash-pool, only some for cache and the rest for fast SSD storage." it's just my (long) experience of this that it's not very productive in reality and it costs a lot of operations (manual work, skilled personnel). It also tends to give you various problems when you need to do HW LCM (upgrade your controllers and disk back ends). It inevitably leads to stranded capacity in more than once dimension as time passes. /M -- Sr Human ;-) Alt: r.m.bergman(a)gmail-DEL_THIS-.com -- "Qui vicit non est victor nisi victus fatetur." - Ennius

3 4

Re: Re: Question about flash pool maximum SSD size and local tiering
by Gislen, Johan 17 Oct '23

17 Oct '23

Hi Florian, If you have the NL-SAS aggregate in place already I’d recommend having a look at using AWA to figure out what the optimal cache size would be: https://docs.netapp.com/us-en/ontap/disks-aggregates/determine-flash-pool-c… That would give you an idea of how much Flash Pool capacity you’d really benefit from. Regarding 3.8TB vs 7.6 TB – if the 7.6 TB drives are not listed as supported in the HWU, it’s most likely not supported. There’s also a note in the HWU for the 3.8TB drives under “System Cache Limits” – “Max number of Flash Pool Data SSDs: 20” (this is excluding RAID parity & hot spares) – if I’m intepreting that note correcly there’d be no point in adding more than one shelf of 3.8TB drives. Regards, Johan On 2023-10-17, 09:57, "Florian Schmid" <fschmid(a)ubimet.com> wrote: Hi Sebastian and Michael, thank you very much for your help. We haven't bought the SSD shelves yet. We don't know, if we should take 2 shelves with 3.8 TB or 1 shelf with 7.6 TB. I tend to one shelf with 7.6 TB... We are using flash-pool already with Raid-4, exactly for saving space. My problem is now, that I'm concerned about the HWU statement, that there is no SSD listed for using in flash-pool over 3.8 TB. This let me think, that flash-pool is not supported for SSD with 7.6 TB or greater. I haven't found anything else about it on NetApp site, only in the HWU for system cache limits for FAS 8300 Same limits are also for 8700 or 9000 series, no difference here. Really strange. Best regards, Florian ----- Ursprüngliche Mail ----- Von: "Sebastian Goetze" <spgoetze(a)gmail.com<mailto:spgoetze@gmail.com>> An: "toasters" <toasters(a)teaparty.net<mailto:toasters@teaparty.net>> Gesendet: Montag, 16. Oktober 2023 15:35:36 Betreff: Re: Question about flash pool maximum SSD size and local tiering Hi Florian, On 16.10.2023 15:01, Fenn, Michael wrote: > FlashPool SSDs (unlike FlashCache) are attached to aggregates as normal RAID groups, so you can use as many 3.8 TB drives in RAID-DP as you like to hit the maximum FlashPool capacity. Consider using RAID4 for the SSD RaidGroups... More Cache, and for the Write-cached blocks, not really less safety. (SSDs are more reliable, blocks are usually written fairly soon to HDD anyway) Regarding sizes: any size that's supported for the HW should be fine (3.8/15.3TB). Within the Cache-RG, you should use the same size disks, however... > > Note that FlashPool and FabricPool use the same underlying tiering metadata structures, so you can only have one or the other enabled on any given aggregate. > > Thanks, > Michael > > On 10/16/23, 8:03 AM, "Toasters on behalf of Florian Schmid via Toasters" <toasters-bounces(a)teaparty.net<mailto:toasters-bounces@teaparty.net> <mailto:toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net>> on behalf of toasters(a)teaparty.net<mailto:toasters@teaparty.net> <mailto:toasters@teaparty.net<mailto:toasters@teaparty.net>>> wrote: > > > Hi Alexander, > > this is a very good tip! Tank you very much. > I will have a look on this. > > Best regards, > Florian > > ----- Ursprüngliche Mail ----- > Von: "Alexander Griesser" <AGriesser(a)anexia.com<mailto:AGriesser@anexia.com>> > An: "Florian Schmid" <fschmid(a)ubimet.com<mailto:fschmid@ubimet.com>>, "toasters" <toasters(a)teaparty.net<mailto:toasters@teaparty.net>> > Gesendet: Montag, 16. Oktober 2023 12:07:53 > Betreff: AW: Question about flash pool maximum SSD size and local tiering > > Hi Florian, > > I cannot answer the question with the SSD sizes, I'm not sure if this is really a hard requirement or if the slices just may not be bigger than 3.8TB (in that case, you could probably manually partition the SSDs), maybe someone else has more insights into this. > > As for your second question: You can spin up OnTap's integrated S3 server on your old boxes and use them as fabric pool targets: > https://www.netapp.com/media/17219-tr4814.pdf > > Best, > > Alexander Griesser > Head of Systems Operations > > ANEXIA Internetdienstleistungs GmbH > > E-Mail: AGriesser(a)anexia.com<mailto:AGriesser@anexia.com> > Web: https://www.anexia.com > > Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt > Geschäftsführer: Alexander Windbichler > Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601 > > -----Ursprüngliche Nachricht----- > Von: Florian Schmid <fschmid(a)ubimet.com<mailto:fschmid@ubimet.com>> > Gesendet: Montag, 16. Oktober 2023 11:53 > An: toasters(a)teaparty.net<mailto:toasters@teaparty.net> > Betreff: Question about flash pool maximum SSD size and local tiering > > Hi, > > I have checked NetApp HWU for a FAS 8300 and system cache limits. > > Ok, so far, max flash-pool is 72 TB, which is a way more than I want to use, but I haven't seen usable SSDs greater than 3.8 TB. > > Is that really true, that I can't use a 7.6 TB or 15.3 TB SSD for flash-pool? > > It would be nice, if someone has here a deeper understanding than I have about this and can give me here some clarifications. > > May I ask a second question? > Is flash-pool still the way to go for speeding up NL-SAS aggregates? > I had a look on fabric-pool tiering, but it seems like, that this only works to S3 storage, which we don't have. > We have plenty of NL-SAS storage and also of SSDs and it would be great to have a tiering between them or at least use them for caching. > > Best regards, > Florian > > > > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net<mailto:Toasters@teaparty.net> > https://www.teaparty.net/mailman/listinfo/toasters _______________________________________________ Toasters mailing list Toasters(a)teaparty.net<mailto:Toasters@teaparty.net> https://www.teaparty.net/mailman/listinfo/toasters

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters