To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance? 2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
Hi Bill,
we have ASIS on two of our 6070's for some time. We did only few tests but could not see any direct impact on the filers performance. We have not yet used it for LUNs so far, but would be interessted in experience with ASIS on LUNs as well.
Regards
Jochen
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 12:50 PM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance? 2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
I would be interested in knowing the impact of a-sis on cache hit % though. Since you're essentially changing pointers to other instances of blocks, those de-duped blocks wouldn't be located along with the other original blocks, so it could affect the performance of onTap's read-ahead algorithm.
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Willeke, Jochen Sent: Wednesday, February 13, 2008 8:01 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Hi Bill,
we have ASIS on two of our 6070's for some time. We did only few tests but could not see any direct impact on the filers performance.
We have not yet used it for LUNs so far, but would be interessted in experience with ASIS on LUNs as well.
Regards
Jochen
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 12:50 PM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
________________________________
Wincor Nixdorf International GmbH Sitz der Gesellschaft: Paderborn Registergericht Paderborn HRB 3507 Geschftsfhrer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr. Jrgen Wunram Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr. DE44477193
Diese E-Mail enthlt vertrauliche Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtmlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.
This e-mail may contain confidential information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
Have you got any pointers regarding this? This is one not quite clear point to me too – whether deduplication affects sequential IO and what “real life” impact is.
С уважением / With best regards / Mit freundlichen Grüβen
--- Andrey Borzenkov Senior system engineer
________________________________ From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Glenn Dekhayser Sent: Wednesday, February 13, 2008 5:29 PM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
I would be interested in knowing the impact of a-sis on cache hit % though. Since you’re essentially changing pointers to other instances of blocks, those de-duped blocks wouldn’t be located along with the other original blocks, so it could affect the performance of onTap’s read-ahead algorithm.
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Willeke, Jochen Sent: Wednesday, February 13, 2008 8:01 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Hi Bill,
we have ASIS on two of our 6070's for some time. We did only few tests but could not see any direct impact on the filers performance. We have not yet used it for LUNs so far, but would be interessted in experience with ASIS on LUNs as well.
Regards
Jochen
________________________________ From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 12:50 PM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance? 2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform. ________________________________ Wincor Nixdorf International GmbH Sitz der Gesellschaft: Paderborn Registergericht Paderborn HRB 3507 Geschftsfhrer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr. Jrgen Wunram Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr. DE44477193
Diese E-Mail enthlt vertrauliche Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtmlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.
This e-mail may contain confidential information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
We've been doing some VMWare testing with FCP LUNs and A-SIS.
We saw a reduction from 471GB to 21GB with only about a 7% reduction in performance. More than a fair trade-off in my opinion.
Our testing could have had impact on the performance more than the de-dupe, however.
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 6:50 AM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
Thanks for the stats, I'll be de-duping VMWare data soon too.
My NetApp storage tech says that read cache peformance will increase since you're reducing the total number of actual blocks. I have not heard of any performance degradations with A-SIS, other than filer overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules are during off-peak hours so it's not a concern of mine.
Daniel
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Glenn Walker Sent: Wednesday, February 13, 2008 9:42 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
We've been doing some VMWare testing with FCP LUNs and A-SIS.
We saw a reduction from 471GB to 21GB with only about a 7% reduction in performance. More than a fair trade-off in my opinion.
Our testing could have had impact on the performance more than the de-dupe, however.
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 6:50 AM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
______________________________________________________________________ This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
Just keep in mind that we were using IOMeter to test, not actual real world workloads - we were also very much disk bound in our testing. If you are neither using IOMeter (ie, a real-world workload) nor disk bound, I can't foresee it being a huge problem.
We'll be testing with NFS dedupe a bit later today and I'll gladly share that info if you want. Same number of disks, so same stipulations exist.
And because the question did come up, the VMWare server guy didn't build the VMDKs with thin provisioning, so it may have also impacted the testing.
Glenn
________________________________
From: Daniel Keisling [mailto:daniel.keisling@austin.ppdi.com] Sent: Wednesday, February 13, 2008 12:37 PM To: Glenn Walker; Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Thanks for the stats, I'll be de-duping VMWare data soon too.
My NetApp storage tech says that read cache peformance will increase since you're reducing the total number of actual blocks. I have not heard of any performance degradations with A-SIS, other than filer overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules are during off-peak hours so it's not a concern of mine.
Daniel
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Glenn Walker Sent: Wednesday, February 13, 2008 9:42 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
We've been doing some VMWare testing with FCP LUNs and A-SIS.
We saw a reduction from 471GB to 21GB with only about a 7% reduction in performance. More than a fair trade-off in my opinion.
Our testing could have had impact on the performance more than the de-dupe, however.
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 6:50 AM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
______________________________________________________________________ This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
Like the others, thanks for the info. At one point, A-SIS wasn't supported in true "production" roles, meaning in a Tier-1 use case. Has that changed recently? If so, I think I need to make some phone calls =).
On the thin provisioning - ESX doesn't support thin-provisioned VMDK's yet. Workstation does that by default, but I'm not sure about VMware Server. Hopefully ESX is coming soon though. I would still imagine a fair amount of reclaimed space once ESX does come through. While you'll no longer de-dupe the white space in a VMDK, I'd imagine you'll still de-dupe all the base OS commonalities.
Jeff Mery - MCSE, MCP National Instruments
------------------------------------------------------------------------- "Allow me to extol the virtues of the Net Fairy, and of all the fantastic dorks that make the nice packets go from here to there. Amen." TB - Penny Arcade -------------------------------------------------------------------------
From: "Glenn Walker" ggwalker@mindspring.com To: "Daniel Keisling" daniel.keisling@austin.ppdi.com, "Bill Holland" hollandwl@gmail.com, toasters@mathworks.com Date: 02/13/2008 12:37 PM Subject: RE: De-dup'ing Primary Storage
Just keep in mind that we were using IOMeter to test, not actual real world workloads – we were also very much disk bound in our testing. If you are neither using IOMeter (ie, a real-world workload) nor disk bound, I can’t foresee it being a huge problem.
We’ll be testing with NFS dedupe a bit later today and I’ll gladly share that info if you want. Same number of disks, so same stipulations exist.
And because the question did come up, the VMWare server guy didn’t build the VMDKs with thin provisioning, so it may have also impacted the testing.
Glenn
From: Daniel Keisling [mailto:daniel.keisling@austin.ppdi.com] Sent: Wednesday, February 13, 2008 12:37 PM To: Glenn Walker; Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Thanks for the stats, I'll be de-duping VMWare data soon too.
My NetApp storage tech says that read cache peformance will increase since you're reducing the total number of actual blocks. I have not heard of any performance degradations with A-SIS, other than filer overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules are during off-peak hours so it's not a concern of mine.
Daniel
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Glenn Walker Sent: Wednesday, February 13, 2008 9:42 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage We’ve been doing some VMWare testing with FCP LUNs and A-SIS.
We saw a reduction from 471GB to 21GB with only about a 7% reduction in performance. More than a fair trade-off in my opinion.
Our testing could have had impact on the performance more than the de-dupe, however.
Glenn
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 6:50 AM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance? 2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
______________________________________________________________________ This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
Supportability for ESX I cannot say, but the NetApp VMWare whitepaper tells you how to set up thin-provisioned VMDKs. Why wouldn't you de-dupe the white space in the VMDK? If all zeros are written, then it should de-dupe just fine ;)
Supportability for A-SIS is another thing I cannot say for sure, but I know it works :-) The only documentation I can find (up to 7.2.4) states something about it only being supported for NetBackup on NetApp.
________________________________
From: jeff.mery@ni.com [mailto:jeff.mery@ni.com] Sent: Wednesday, February 13, 2008 2:48 PM To: Glenn Walker Cc: Daniel Keisling; Bill Holland; owner-toasters@mathworks.com; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Like the others, thanks for the info. At one point, A-SIS wasn't supported in true "production" roles, meaning in a Tier-1 use case. Has that changed recently? If so, I think I need to make some phone calls =).
On the thin provisioning - ESX doesn't support thin-provisioned VMDK's yet. Workstation does that by default, but I'm not sure about VMware Server. Hopefully ESX is coming soon though. I would still imagine a fair amount of reclaimed space once ESX does come through. While you'll no longer de-dupe the white space in a VMDK, I'd imagine you'll still de-dupe all the base OS commonalities.
Jeff Mery - MCSE, MCP National Instruments
------------------------------------------------------------------------ - "Allow me to extol the virtues of the Net Fairy, and of all the fantastic dorks that make the nice packets go from here to there. Amen." TB - Penny Arcade ------------------------------------------------------------------------ -
From:
"Glenn Walker" ggwalker@mindspring.com
To:
"Daniel Keisling" daniel.keisling@austin.ppdi.com, "Bill Holland" hollandwl@gmail.com, toasters@mathworks.com
Date:
02/13/2008 12:37 PM
Subject:
RE: De-dup'ing Primary Storage
________________________________
Just keep in mind that we were using IOMeter to test, not actual real world workloads - we were also very much disk bound in our testing. If you are neither using IOMeter (ie, a real-world workload) nor disk bound, I can't foresee it being a huge problem.
We'll be testing with NFS dedupe a bit later today and I'll gladly share that info if you want. Same number of disks, so same stipulations exist.
And because the question did come up, the VMWare server guy didn't build the VMDKs with thin provisioning, so it may have also impacted the testing.
Glenn
________________________________
From: Daniel Keisling [mailto:daniel.keisling@austin.ppdi.com mailto:daniel.keisling@austin.ppdi.com ] Sent: Wednesday, February 13, 2008 12:37 PM To: Glenn Walker; Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Thanks for the stats, I'll be de-duping VMWare data soon too.
My NetApp storage tech says that read cache peformance will increase since you're reducing the total number of actual blocks. I have not heard of any performance degradations with A-SIS, other than filer overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules are during off-peak hours so it's not a concern of mine.
Daniel
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com mailto:owner-toasters@mathworks.com ] On Behalf Of Glenn Walker Sent: Wednesday, February 13, 2008 9:42 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage We've been doing some VMWare testing with FCP LUNs and A-SIS.
We saw a reduction from 471GB to 21GB with only about a 7% reduction in performance. More than a fair trade-off in my opinion.
Our testing could have had impact on the performance more than the de-dupe, however.
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com mailto:owner-toasters@mathworks.com ] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 6:50 AM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance? 2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
______________________________________________________________________ This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
Hi Jeff! Make your calls! ;-)
ESX does support thin-provisioned VMDKs. They are default in NFS datastores. You have to use vmkfstools from the command line to create thin-provisioned VMDKs in VMFS. Either way, when you clone a thin VM in VirtualCenter, it makes a thick clone.
There is also thin provisioning on the NetApp side, which some customers are also using successfully (at least one that I know of for over a year).
Enjoy!
Peter
________________________________
From: jeff.mery@ni.com [mailto:jeff.mery@ni.com] Sent: Wednesday, February 13, 2008 11:48 AM To: Glenn Walker Cc: Daniel Keisling; Bill Holland; owner-toasters@mathworks.com; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Like the others, thanks for the info. At one point, A-SIS wasn't supported in true "production" roles, meaning in a Tier-1 use case. Has that changed recently? If so, I think I need to make some phone calls =).
On the thin provisioning - ESX doesn't support thin-provisioned VMDK's yet. Workstation does that by default, but I'm not sure about VMware Server. Hopefully ESX is coming soon though. I would still imagine a fair amount of reclaimed space once ESX does come through. While you'll no longer de-dupe the white space in a VMDK, I'd imagine you'll still de-dupe all the base OS commonalities.
Jeff Mery - MCSE, MCP National Instruments
------------------------------------------------------------------------ - "Allow me to extol the virtues of the Net Fairy, and of all the fantastic dorks that make the nice packets go from here to there. Amen." TB - Penny Arcade ------------------------------------------------------------------------ -
From: "Glenn Walker" ggwalker@mindspring.com To: "Daniel Keisling" daniel.keisling@austin.ppdi.com, "Bill Holland" hollandwl@gmail.com, toasters@mathworks.com Date: 02/13/2008 12:37 PM Subject: RE: De-dup'ing Primary Storage
________________________________
Just keep in mind that we were using IOMeter to test, not actual real world workloads - we were also very much disk bound in our testing. If you are neither using IOMeter (ie, a real-world workload) nor disk bound, I can't foresee it being a huge problem.
We'll be testing with NFS dedupe a bit later today and I'll gladly share that info if you want. Same number of disks, so same stipulations exist.
And because the question did come up, the VMWare server guy didn't build the VMDKs with thin provisioning, so it may have also impacted the testing.
Glenn
________________________________
From: Daniel Keisling [mailto:daniel.keisling@austin.ppdi.com mailto:daniel.keisling@austin.ppdi.com ] Sent: Wednesday, February 13, 2008 12:37 PM To: Glenn Walker; Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Thanks for the stats, I'll be de-duping VMWare data soon too.
My NetApp storage tech says that read cache peformance will increase since you're reducing the total number of actual blocks. I have not heard of any performance degradations with A-SIS, other than filer overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules are during off-peak hours so it's not a concern of mine.
Daniel
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com mailto:owner-toasters@mathworks.com ] On Behalf Of Glenn Walker Sent: Wednesday, February 13, 2008 9:42 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage We've been doing some VMWare testing with FCP LUNs and A-SIS.
We saw a reduction from 471GB to 21GB with only about a 7% reduction in performance. More than a fair trade-off in my opinion.
Our testing could have had impact on the performance more than the de-dupe, however.
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com mailto:owner-toasters@mathworks.com ] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 6:50 AM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance? 2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
______________________________________________________________________ This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
FYI - we've just completed testing with NFS and dedupe - about 72% savings (VMDK on NFS is thin provisioned by default anyways, so no big shock there). About 4% performance reduction, easily acceptable.
One thing to keep in mind - the busier the drives, the sharper the performance loss (as would be expected): We only have a 21 disk I/O pool (14D+2P, 7D+2P AGGR), and we're getting about 38MB/s for an 8K transfer size, 100% random, 25%/75% r/w mix with a 4GB test file per VM Guest - 12 guests total. Disks are almost maxed out (95% avg I'd say - disks are upsetting the NVRAM pretty quickly) and adding de-dupe dropped it to 36.8MB/s. Kicking it up to 15 guests creates 100% disk I/O and adding de-dupe gives about a 25% performance loss. Testing with 2MB transfer gives us about 112MB/s, so the testing is pretty subjective - as is the performance loss.
The thing to keep in mind: performance will _always_ suck when 100% disk utilization kicks in. The 4% performance loss we've seen for NFS w/ de-dupe and 7% performance loss with FCP de-dupe when disks are almost completely at max are cases where more disk I/O would help and I'd guess that we'd see little to no performance degradation.
We've found that the performance curve for NFS and FCP dedupe are about the same - the performance for both for a VMWare environment are pretty close as well.
Hope this info helps some of you...
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Glenn Walker Sent: Wednesday, February 13, 2008 12:50 PM To: Daniel Keisling; Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Just keep in mind that we were using IOMeter to test, not actual real world workloads - we were also very much disk bound in our testing. If you are neither using IOMeter (ie, a real-world workload) nor disk bound, I can't foresee it being a huge problem.
We'll be testing with NFS dedupe a bit later today and I'll gladly share that info if you want. Same number of disks, so same stipulations exist.
And because the question did come up, the VMWare server guy didn't build the VMDKs with thin provisioning, so it may have also impacted the testing.
Glenn
________________________________
From: Daniel Keisling [mailto:daniel.keisling@austin.ppdi.com] Sent: Wednesday, February 13, 2008 12:37 PM To: Glenn Walker; Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
Thanks for the stats, I'll be de-duping VMWare data soon too.
My NetApp storage tech says that read cache peformance will increase since you're reducing the total number of actual blocks. I have not heard of any performance degradations with A-SIS, other than filer overhead (CPU) when SIS is actually de-duplicating. My A-SIS schedules are during off-peak hours so it's not a concern of mine.
Daniel
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Glenn Walker Sent: Wednesday, February 13, 2008 9:42 AM To: Bill Holland; toasters@mathworks.com Subject: RE: De-dup'ing Primary Storage
We've been doing some VMWare testing with FCP LUNs and A-SIS.
We saw a reduction from 471GB to 21GB with only about a 7% reduction in performance. More than a fair trade-off in my opinion.
Our testing could have had impact on the performance more than the de-dupe, however.
Glenn
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Bill Holland Sent: Wednesday, February 13, 2008 6:50 AM To: toasters@mathworks.com Subject: De-dup'ing Primary Storage
To those of you that have implemented Nearstore and A-SIS on your primary storage:
1. Have you seen any difference in overall filer performance?
2. If you have LUNs, how are your space savings on those volumes?
I know that enabling Nearstore does some system tweaking in the background to increase the number of concurrent backup streams that can be running, but I don't know what else it tweaks that may adversely affect performance of a primary storage system. Afterall, it was originally designed to run as a secondary storage platform.
______________________________________________________________________ This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
-
Bill Holland -
Borzenkov, Andrey -
Daniel Keisling -
Glenn Dekhayser -
Glenn Walker -
jeff.mery＠ni.com -
Learmonth, Peter -
Willeke, Jochen