So I'm looking at upgrading a few 8020s and trying to choose between flashpool and flashcache. Flashpool is more expensive, even with just 12x200 SSD drives, and we'd need to find space in a very tight rack for it. The workload is 80% exchange and 20% VMWare, and we would like to use deduplication without snapshots.
Is it possible to use PCS to compare flashcache and flashpool benefits? How can I translate the data I see from a stats show -p flexscale-access into real numbers, like reduction of latency?
Basil -
I would be very careful with choosing in this case. FlexPool may be marginally more expensive but seeing as how the 8020 only has two PCIe slots adding a FlashCache module will severely limit your expansion options. If that other slot is occupied by another 10GbE card you'll be limited to one SAS stack (10 shelves). Furthermore, per best practices, mixing SAS/SATA drive types on the same stack is not encouraged (although it can be done), so you're essentially limiting yourself to one drive type. On the other hand, if the other slot is occupied by a SAS card then you're limiting your connectivity options. If you're running CDOT you're already going to be giving up two 10GbE onboard ports for the cluster interconnect so effectively you have two 10GbE/FC ports per controller. If in the future you decided you wanted to add a tape drive (FC) or more 10GbE ports you'd be out of luck.
Regarding performance, PCS doesn't really differentiate, and to be honest for most real world situations I've seen it's a wash. The ability to have write caching via FlashPool is very helpful for random overwrite workloads. 'wafl awa' can help you determine the impact adding cache of either kind will have on writes/reads and how much optimally you would need. It is important to run it at a peak time to see what the recommendations are, as running it during a relatively low activity period won't give much useful information.
Anthony Bar 650.207.5368tel:650.207.5368 tbar@berkcom.commailto:tbar@berkcom.com
On Jul 3, 2015, at 10:03 AM, Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com> wrote:
So I'm looking at upgrading a few 8020s and trying to choose between flashpool and flashcache. Flashpool is more expensive, even with just 12x200 SSD drives, and we'd need to find space in a very tight rack for it. The workload is 80% exchange and 20% VMWare, and we would like to use deduplication without snapshots.
Is it possible to use PCS to compare flashcache and flashpool benefits? How can I translate the data I see from a stats show -p flexscale-access into real numbers, like reduction of latency? _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Thanks for the note about the card slots- I'm checking to see whether we even have a spare. Also, that wafl awa command is precisely what I was looking for, thanks!
On Fri, Jul 3, 2015 at 2:57 PM, Tony Bar tbar@berkcom.com wrote:
Basil -
I would be very careful with choosing in this case. FlexPool may be marginally more expensive but seeing as how the 8020 only has two PCIe slots adding a FlashCache module will severely limit your expansion options. If that other slot is occupied by another 10GbE card you'll be limited to one SAS stack (10 shelves). Furthermore, per best practices, mixing SAS/SATA drive types on the same stack is not encouraged (although it can be done), so you're essentially limiting yourself to one drive type. On the other hand, if the other slot is occupied by a SAS card then you're limiting your connectivity options. If you're running CDOT you're already going to be giving up two 10GbE onboard ports for the cluster interconnect so effectively you have two 10GbE/FC ports per controller. If in the future you decided you wanted to add a tape drive (FC) or more 10GbE ports you'd be out of luck.
Regarding performance, PCS doesn't really differentiate, and to be honest for most real world situations I've seen it's a wash. The ability to have write caching via FlashPool is very helpful for random overwrite workloads. 'wafl awa' can help you determine the impact adding cache of either kind will have on writes/reads and how much optimally you would need. It is important to run it at a peak time to see what the recommendations are, as running it during a relatively low activity period won't give much useful information.
Anthony Bar 650.207.5368 tbar@berkcom.com
On Jul 3, 2015, at 10:03 AM, Basil basilberntsen@gmail.com wrote:
So I'm looking at upgrading a few 8020s and trying to choose between flashpool and flashcache. Flashpool is more expensive, even with just 12x200 SSD drives, and we'd need to find space in a very tight rack for it. The workload is 80% exchange and 20% VMWare, and we would like to use deduplication without snapshots.
Is it possible to use PCS to compare flashcache and flashpool benefits? How can I translate the data I see from a stats show -p flexscale-access into real numbers, like reduction of latency?
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
As FlashPool & CIFS are (currently) mutually exclusive I can only see FlashCache in CIFS environments in the future. The Netapp FlashPool sizing suggestions are a good place to start and you can start small, keep an eye on the stats and avoid diminishing returns. cDOT 8.3 supports partitions and adding partitions is fine though removing them is not somewhere you want to go!
Speaking of future, the point at which it becomes more expensive to assemble and provide a spinning disk assembly than a similar high-capacity SSD will be an overnight game-changer consigning these arguments to the archives. Any predictions on year/month? Sweepstake anyone? I’m not sure that the industry really wants to see this happen, so make take a while ;-) Neil
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Basil Sent: Saturday, 4 July 2015 7:31 a.m. To: Tony Bar Cc: toasters@teaparty.net Subject: Re: Another SSD question
Thanks for the note about the card slots- I'm checking to see whether we even have a spare. Also, that wafl awa command is precisely what I was looking for, thanks!
On Fri, Jul 3, 2015 at 2:57 PM, Tony Bar <tbar@berkcom.commailto:tbar@berkcom.com> wrote: Basil -
I would be very careful with choosing in this case. FlexPool may be marginally more expensive but seeing as how the 8020 only has two PCIe slots adding a FlashCache module will severely limit your expansion options. If that other slot is occupied by another 10GbE card you'll be limited to one SAS stack (10 shelves). Furthermore, per best practices, mixing SAS/SATA drive types on the same stack is not encouraged (although it can be done), so you're essentially limiting yourself to one drive type. On the other hand, if the other slot is occupied by a SAS card then you're limiting your connectivity options. If you're running CDOT you're already going to be giving up two 10GbE onboard ports for the cluster interconnect so effectively you have two 10GbE/FC ports per controller. If in the future you decided you wanted to add a tape drive (FC) or more 10GbE ports you'd be out of luck.
Regarding performance, PCS doesn't really differentiate, and to be honest for most real world situations I've seen it's a wash. The ability to have write caching via FlashPool is very helpful for random overwrite workloads. 'wafl awa' can help you determine the impact adding cache of either kind will have on writes/reads and how much optimally you would need. It is important to run it at a peak time to see what the recommendations are, as running it during a relatively low activity period won't give much useful information.
Anthony Bar 650.207.5368tel:650.207.5368 tbar@berkcom.commailto:tbar@berkcom.com
On Jul 3, 2015, at 10:03 AM, Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com> wrote: So I'm looking at upgrading a few 8020s and trying to choose between flashpool and flashcache. Flashpool is more expensive, even with just 12x200 SSD drives, and we'd need to find space in a very tight rack for it. The workload is 80% exchange and 20% VMWare, and we would like to use deduplication without snapshots.
Is it possible to use PCS to compare flashcache and flashpool benefits? How can I translate the data I see from a stats show -p flexscale-access into real numbers, like reduction of latency? _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Flash Pool and CIFS absolutely work together.
Chris
From: <toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net> on behalf of Neil Stichbury <Neil.Stichbury@spark.co.nzmailto:Neil.Stichbury@spark.co.nz> Date: Saturday, July 4, 2015 at 8:04 PM To: Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com>, "NGC-tbar-berkcom.com" <tbar@berkcom.commailto:tbar@berkcom.com> Cc: "<toasters@teaparty.netmailto:toasters@teaparty.net> Lists" <toasters@teaparty.netmailto:toasters@teaparty.net> Subject: RE: Another SSD question
As FlashPool & CIFS are (currently) mutually exclusive I can only see FlashCache in CIFS environments in the future. The Netapp FlashPool sizing suggestions are a good place to start and you can start small, keep an eye on the stats and avoid diminishing returns. cDOT 8.3 supports partitions and adding partitions is fine though removing them is not somewhere you want to go!
Speaking of future, the point at which it becomes more expensive to assemble and provide a spinning disk assembly than a similar high-capacity SSD will be an overnight game-changer consigning these arguments to the archives. Any predictions on year/month? Sweepstake anyone? I'm not sure that the industry really wants to see this happen, so make take a while ;-) Neil
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Basil Sent: Saturday, 4 July 2015 7:31 a.m. To: Tony Bar Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: Another SSD question
Thanks for the note about the card slots- I'm checking to see whether we even have a spare. Also, that wafl awa command is precisely what I was looking for, thanks!
On Fri, Jul 3, 2015 at 2:57 PM, Tony Bar <tbar@berkcom.commailto:tbar@berkcom.com> wrote: Basil -
I would be very careful with choosing in this case. FlexPool may be marginally more expensive but seeing as how the 8020 only has two PCIe slots adding a FlashCache module will severely limit your expansion options. If that other slot is occupied by another 10GbE card you'll be limited to one SAS stack (10 shelves). Furthermore, per best practices, mixing SAS/SATA drive types on the same stack is not encouraged (although it can be done), so you're essentially limiting yourself to one drive type. On the other hand, if the other slot is occupied by a SAS card then you're limiting your connectivity options. If you're running CDOT you're already going to be giving up two 10GbE onboard ports for the cluster interconnect so effectively you have two 10GbE/FC ports per controller. If in the future you decided you wanted to add a tape drive (FC) or more 10GbE ports you'd be out of luck.
Regarding performance, PCS doesn't really differentiate, and to be honest for most real world situations I've seen it's a wash. The ability to have write caching via FlashPool is very helpful for random overwrite workloads. 'wafl awa' can help you determine the impact adding cache of either kind will have on writes/reads and how much optimally you would need. It is important to run it at a peak time to see what the recommendations are, as running it during a relatively low activity period won't give much useful information.
Anthony Bar 650.207.5368tel:650.207.5368 tbar@berkcom.commailto:tbar@berkcom.com
On Jul 3, 2015, at 10:03 AM, Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com> wrote: So I'm looking at upgrading a few 8020s and trying to choose between flashpool and flashcache. Flashpool is more expensive, even with just 12x200 SSD drives, and we'd need to find space in a very tight rack for it. The workload is 80% exchange and 20% VMWare, and we would like to use deduplication without snapshots.
Is it possible to use PCS to compare flashcache and flashpool benefits? How can I translate the data I see from a stats show -p flexscale-access into real numbers, like reduction of latency? _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Apologies, yes. Tony Bar succinctly stated what I intended. Regards, Neil
From: Waltham, Christopher [mailto:Christopher.Waltham@netapp.com] Sent: Tuesday, 7 July 2015 2:24 a.m. To: Neil Stichbury; Basil; NGC-tbar-berkcom.com Cc: toasters@teaparty.net Lists Subject: Re: Another SSD question
Flash Pool and CIFS absolutely work together.
Chris
From: <toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net> on behalf of Neil Stichbury <Neil.Stichbury@spark.co.nzmailto:Neil.Stichbury@spark.co.nz> Date: Saturday, July 4, 2015 at 8:04 PM To: Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com>, "NGC-tbar-berkcom.com" <tbar@berkcom.commailto:tbar@berkcom.com> Cc: "<toasters@teaparty.netmailto:toasters@teaparty.net> Lists" <toasters@teaparty.netmailto:toasters@teaparty.net> Subject: RE: Another SSD question
As FlashPool & CIFS are (currently) mutually exclusive I can only see FlashCache in CIFS environments in the future. The Netapp FlashPool sizing suggestions are a good place to start and you can start small, keep an eye on the stats and avoid diminishing returns. cDOT 8.3 supports partitions and adding partitions is fine though removing them is not somewhere you want to go!
Speaking of future, the point at which it becomes more expensive to assemble and provide a spinning disk assembly than a similar high-capacity SSD will be an overnight game-changer consigning these arguments to the archives. Any predictions on year/month? Sweepstake anyone? I'm not sure that the industry really wants to see this happen, so make take a while ;-) Neil
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Basil Sent: Saturday, 4 July 2015 7:31 a.m. To: Tony Bar Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: Another SSD question
Thanks for the note about the card slots- I'm checking to see whether we even have a spare. Also, that wafl awa command is precisely what I was looking for, thanks!
On Fri, Jul 3, 2015 at 2:57 PM, Tony Bar <tbar@berkcom.commailto:tbar@berkcom.com> wrote: Basil -
I would be very careful with choosing in this case. FlexPool may be marginally more expensive but seeing as how the 8020 only has two PCIe slots adding a FlashCache module will severely limit your expansion options. If that other slot is occupied by another 10GbE card you'll be limited to one SAS stack (10 shelves). Furthermore, per best practices, mixing SAS/SATA drive types on the same stack is not encouraged (although it can be done), so you're essentially limiting yourself to one drive type. On the other hand, if the other slot is occupied by a SAS card then you're limiting your connectivity options. If you're running CDOT you're already going to be giving up two 10GbE onboard ports for the cluster interconnect so effectively you have two 10GbE/FC ports per controller. If in the future you decided you wanted to add a tape drive (FC) or more 10GbE ports you'd be out of luck.
Regarding performance, PCS doesn't really differentiate, and to be honest for most real world situations I've seen it's a wash. The ability to have write caching via FlashPool is very helpful for random overwrite workloads. 'wafl awa' can help you determine the impact adding cache of either kind will have on writes/reads and how much optimally you would need. It is important to run it at a peak time to see what the recommendations are, as running it during a relatively low activity period won't give much useful information.
Anthony Bar 650.207.5368tel:650.207.5368 tbar@berkcom.commailto:tbar@berkcom.com
On Jul 3, 2015, at 10:03 AM, Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com> wrote: So I'm looking at upgrading a few 8020s and trying to choose between flashpool and flashcache. Flashpool is more expensive, even with just 12x200 SSD drives, and we'd need to find space in a very tight rack for it. The workload is 80% exchange and 20% VMWare, and we would like to use deduplication without snapshots.
Is it possible to use PCS to compare flashcache and flashpool benefits? How can I translate the data I see from a stats show -p flexscale-access into real numbers, like reduction of latency? _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Tony Bar wrote:
'wafl awa' can help you determine the impact adding cache of either kind will have on writes/reads and how much optimally you would need. It is important to run it at a peak time to see what the recommendations are, as running it during a relatively low activity period won't give much useful information.
The best is really to let it run for several days, preferrably so that it fills up its DB of samples (1000 max, each is 600 s by default). Once it's done that, you're at stable state so to speak and you can trust what it says much more
Here's a sample output just to show you from one of our hard working NFS/CIFS Filers (no Block Storage traffic at all), it's a 8060c. Note that this system already has FlashPool (quite a lot too) on the only Aggr in each head. AWA will do its work anyway, suggesting if you should add more SSDs (provided you have the dough ;-) ) or if you've spent too much previously. N.B. AWA assumes the default policy everywhere: random_read-random_write
### FP AWA Stats ###
Host sekinaXXX Memory 61044 MB ONTAP Version NetApp Release 8.2.3P2D10 7-Mode: Thu Apr 9 22:11:11 PDT 2015 AWA Version 1 Layout Version 1 CM Version 1
Memory Allocation Memory Used 69848408 Malloc Count 16692 Mfree Count 29
Basic Information
Aggregate aggr0 Current-time Tue Jul 7 16:31:16 CEST 2015 Start-time Tue Jun 30 17:51:58 CEST 2015 Total runtime (sec) 599965 Interval length (sec) 600 Total intervals 1000 In-core Intervals 1024
Summary of the past 1000 intervals max Read Throughput 260.129 MB/s Write Throughput 172.245 MB/s Cacheable Read (%) 50 % Cacheable Write (%) 23 % Max Projected Cache Size 1251 GiB Projected Read Offload 68 % Projected Write Offload 22 %
Summary Cache Hit Rate vs. Cache Size
Size 20% 40% 60% 80% 100% Read Hit 61 62 63 67 68 Write Hit 19 19 21 21 22
The entire results and output of Automated Workload Analyzer (AWA) are estimates. The format, syntax, CLI, results and output of AWA may change in future Data ONTAP releases. AWA reports the projected cache size in capacity. It does not make recommendations regarding the number of data SSDs required. Please follow the guidelines for configuring and deploying Flash Pool; that are provided in tools and collateral documents. These include verifying the platform cache size maximums and minimum number and maximum number of data SSDs.
-
Basil -
Michael Bergman -
Neil Stichbury -
Tony Bar -
Waltham, Christopher