I'm currently contemplating caching on my filers. Specifically - setting those 3 flags on a flexscale cache appropriately.
I _think_ given the workload I'm putting through one of my set of filers, enabling lopri_blocks is beneficial
When I do, a 'stats show -p flexscale-access' gives me more disk reads replaced, and a better cache hit ratio.
However I'm also looking at wafl:wafl:read_io_type and seeing that mostly - my cache module is only servicing about 10% of the IOPs going through my system anyway. Of the remainder, it's a pretty even split between disk/cache.
And with 'lopri_blocks' set to on, my cache is churning data faster - but with it off, hit rate drops to <10%.
I'm wondering if anyone can offer suggestions as to a good way to compare the two states - filer side. Are there any metrics I _should_ be looking at, that I'm not?
Primarily looking at: - total IOPs (there's only one volume on this filer) - response time. - Disk utilisation (definitely increases when lopri_blocks is off). - cache stats as above.
CP types - mostly this system is doing log_full CPs, and is getting a fairly steady stream of RW IOPs. Most IOs are NFS getattr and lookups though, which are mostly metadata. Of 100K iops, it's about 90% non-RW and 5% read, 5% write.
I think I'm right in saying that these metadata IOs are going to be fairly small and thus served out of RAM (mostly) anyway.
I'm also (re) reading TR-3832 - Flash Cache best practice. This is a little more vague than I'd like on figuring out whether 'lopri_blocks' will actually be useful.
I _think_ they are based on a better hit rate, and a better reads-replaced rate. Any read replaced is one that's served from a faster tier, and is therefore 'better' objectively, right?