On 2023-10-18 19:28, Jeffrey Steiner wrote:
FlashPool was almost miraculous in its day, and it's still important. I've seen a bit of a resurgence for FlashPool in the past year for similar reasons to what you seem to have. We see these massive archival systems, and I'm strongly recommending generous FlashPool so whatever random IO might happen will be captured by the SSD layer.
For random R IOPS, a big FlashCache will do the job just as well. Arguably better even, depending on the workload (the two are caching at different levels, FlashCache is a victim cache underneath the WAFL buffer cache whereas FlashPool is very different from that).
Sure you can have bigger FlashPool than -Cache, but in reality it will not matter that's my experience of having lots of unstructured file data "archive type" in very large NL-SAS Aggrs with FlashPool for a large no of years.
If you have a lot of random overwrite in your workload (>> 10%) then FlashPool will win, when there's 7.2k rpm drives in the back end.
YMMV but when testing things with AWA and measuring in real production, it's not easy to get a large aggregated file storage workload coming into a 7.2k rpm based ONTAP Aggr to perform well most of the time even if you have a big FlashPool. No matter how you tune it (I did try), it tends to not be used (filled up) as much as you'd expect and more IOPS go to spinning more often than you'd like. (E.g. our 8 TB FlashPool per Aggr was never ever filled to more than 20-30% so there was a waste there; stranded capacity)
It's different of course if you have a well defined application and its behaviour is known. Ideally the working set size and its temporal locality needs to be such that it "suits" how FlashPool works to leverage a large FlashPool size. How to match this is beyond me to be honest, very few NetApp customers would be even close to knowing any of these things about their workloads.
All this said: it's MUCH MUCH better to have a "too large" FlashPool/-Cache than nothing on a 7.2K rpm based Aggr!
The difficulty is to not overspend on SSD's in this scenario, because NetApp's price model makes SSD shelves very very expensive.
N.B. I'm not experienced at all with workloads coming from databases. For that stuff you'd all be wise to listen to Jeff ;-)
I agree that looking at Cx00 is a good idea here in this use case and then leverage FabricPool in a smart way. I also concur here:
"...but there are also some huge capacity projects where that [C-series, large QLC Flash] doesn't quite make financial sense."
Depending on your definition of huge, but let's say PiB scale. Today and for the foreseeable future (5 y) there's no way Flash will be able to compete with large spinning 7.2K rpm NL-SAS in terms of $/(TiB*month). Perhaps not even in the next 10 y.
And the cheapest for true archiving use cases is still tape. To this day. I don't expect this to change soon either.
/M
Jeffrey Steiner wrote:
I spent years building database setups, and if I could get 5% of the total dataset size in the form of FlashPool SSD, then virtually all the IOPS would hit that SSD layer. There was often barely any difference between all-flash and Flashpool configurations. There would still be a lot of IO hitting the spinning drives, but it was the sequential IO, which honestly doesn't benefit much from SSD anyway.
That approach mostly went out the window because all-flash got affordable. Even if you didn't technically need all-flash at the moment, it was cheap enough and futureproof. A second reason is the size of spinning drives. We used to regularly sell systems with eight SSDs and 500 spinning drives. There was a decent amount of spinning disk IOPS to go around. These days, you're often buying dramatically fewer spinning drives, which means it's easier to push them to their collective IOPS limits. FlashPool can be a nice cushion against IO surges.
I'd also recommend taking a look at C-Series. The whole point of C-Series is all-flash capacity projects. It's the natural upgrade path for hybrid SSD systems. I don't know what price looks like. Some customers are definitely swapping hybrid for C-Series, but there are also some huge capacity projects where that doesn't quite make financial sense.
Someday, though. Someday there will be no hybrid spinning disk systems and it will all be on these capacity-based all-flash systems, but there is still a role for FlashPool at present.