Hi Michael,
wow, thank you very much for your time writing this very detailed explanation!
It will be one 2-node 8300 cluster, switchless. The cluster will be mainly used for long time archive storage until it is going to tape or for tape restores. For this, we want to take a huge amount of NL-SAS drives.
I thought for speeding the NL-SAS aggregates a little bit up, we also use some SSDs as flash-pool, like we have it now on our old dev NetApp cluster.
The other SSDs will be used for backup and DR purposes. We have a full production all-flash cluster for our normal workloads.
We think about moving some data to the 8300 cluster in the long term, because not all volumes we have now on SSD must be on flash and might consume there too much "expensive" space.
I will also have a deeper look on fabricpool. I had a look already on it in the past, but as I read S3 storage, I haven't looked deeper into it, as we are not using S3 at all in the moment. This was some years ago. As we always had only one all-flash cluster, I haven't thought about it.
Should fabricpool not also work on a 2-node cluster? So instead of using some SSDs for flashpool, we could create an aggregate on SSD and one on NL-SAS and use the NL-SAS one for S3 storage and then for local fabricpool?
Best regards, Florian
----- Ursprüngliche Mail ----- Von: "Michael Bergman" michael.bergman@norsborg.net An: "Toasters" toasters@teaparty.net Gesendet: Dienstag, 17. Oktober 2023 15:37:35 Betreff: Re: Question about flash pool maximum SSD size and local tiering
On 2023-10-17 14:54, Florian Schmid via Toasters wrote:
Hi Johan, thank you very much for your help.
No, we don't have the disks yet for which the flash-pool should be used. Not all SSDs will be used for flash-pool, only some for cache and the rest for fast SSD storage.
So you're thinking to have several different "physical tiers" with different characteristics (performance, inherent latency) for different workloads -- in the same HA-pair? Several different Aggrs with differing performance and behaviour in the same node, FAS8300? (it's a fairly powerful machine so it can do this adequately in many smaller workload cases).
Or do you mean in different 8300 nodes in a X-node cluster (what's X?)
This idea is much harder to make successful than you probably think. It requires you to know very much about your workloads, your applications, what they do so that you can place the correct data in the right place and you have to have the ability to do this over time as data volumes grow. Assuming they do... It's very hard indeed to automate so you need people who can baby watch this continuously and move data around. Yes that's mostly non-disruptive, but it's still quite a lot of work.
It also pretty much assumes for it to be successful in the longer run that your applications do not change their workload patterns and/or pressure more than very slowly. Is this the case?
All in all FabricPool is much much more automatic. It just does the job itself, pretty much w/o fuss once you've tuned it a bit w.r.t. cool down period(s) and things. It "just works". You do need an S3 target system, but as has already been pointed out it can be ONTAP with NL-SAS drives, if you already have a bunch of these lying about you can repurpose those and instead use new Cx00 (or Ax00) nodes in the "front end". The challenge with FabricPool is the network: the connection between the front end and the S3 back end needs to be very good and solid. You have to understand it fully and know every detail how it's built so you know you can trust it's capacity and latency; traffic can be quite bursty.
I'm not very positive to your idea here I'm afraid:
"Not all SSDs will be used for flash-pool, only some for cache and the rest for fast SSD storage."
it's just my (long) experience of this that it's not very productive in reality and it costs a lot of operations (manual work, skilled personnel). It also tends to give you various problems when you need to do HW LCM (upgrade your controllers and disk back ends). It inevitably leads to stranded capacity in more than once dimension as time passes.
/M