Hi all,
I've recently attended an official event where people from NetApp in few words said:
"DOT 8 7-mode has done its time, it's over, it has been a great os but It's now time to move and push DOT 8 Cluster Mode, everytime, everywhere..." and this also if with a simple two node HA system, including 2220!
What do you think about? Also let us assume that cluster mode can (I add 'theoretically') do all what 7 mode does, don't you think that is quite premature to trumpet this out?
Regards,
They can say that all they want but I think NA has had way too many missteps with C-mode to be pushing it down anyone's
Sent from my iPhone
On Jun 17, 2012, at 10:24 AM, "Milazzo Giacomo" <G.Milazzo@sinergy.itmailto:G.Milazzo@sinergy.it> wrote:
Hi all,
I’ve recently attended an official event where people from NetApp in few words said:
“DOT 8 7-mode has done its time, it’s over, it has been a great os but It’s now time to move and push DOT 8 Cluster Mode, everytime, everywhere…” and this also if with a simple two node HA system, including 2220!
What do you think about? Also let us assume that cluster mode can (I add ‘theoretically’) do all what 7 mode does, don’t you think that is quite premature to trumpet this out?
Regards,
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Well, to be honest, I think it is time to (at least) start thinking about moving to cluster mode. A FAS2220 is a bit on the positive side as you can't do SAN with it in C-Mode, but a 2240 can, so then you are fine. I've been testing quite extensive with the simulator and it actually works fine. A bit to my surprise as well I must admit, I was just as skeptical as you all. The concepts are great and the VServer virtualization layer is a great addition to the WAFL storage virtualization layer. VServers work so much better than vFilers and with c-mode you get all the scalability advantages you need.
So, I'm not surprised all the commercial people at NetApp and partners are pushing to C-Mode. If you have a technology refresh in a few years, you should definitely consider it. If you have one this year, I'm not really sure yet. Not because not all features are supported in c-mode (yet), but right now it is quite a steep learning curve if you're used to 7-mode. You need to re-think most of your storage setup but once you do, I think c-mode will fit a lot of environments.
Bottom line: start reading and testing, I think c-mode actually is a good product.
Regards,
Anton
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blackmor, Chris Sent: zondag 17 juni 2012 17:45 To: Milazzo Giacomo Cc: toasters@teaparty.net Subject: Re: Cluster mode - Market vs field
They can say that all they want but I think NA has had way too many missteps with C-mode to be pushing it down anyone's
Sent from my iPhone
On Jun 17, 2012, at 10:24 AM, "Milazzo Giacomo" <G.Milazzo@sinergy.itmailto:G.Milazzo@sinergy.it> wrote: Hi all,
I've recently attended an official event where people from NetApp in few words said:
"DOT 8 7-mode has done its time, it's over, it has been a great os but It's now time to move and push DOT 8 Cluster Mode, everytime, everywhere..." and this also if with a simple two node HA system, including 2220!
What do you think about? Also let us assume that cluster mode can (I add 'theoretically') do all what 7 mode does, don't you think that is quite premature to trumpet this out?
Regards,
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
This e-mail is personal. For our full disclaimer, please visit www.centric.eu/disclaimer.
We've been wanting C-Mode for a while but there are some standard features that we rely on that aren't there yet. This email got cut off so I sent another. My basic thought on it is - sure, there are those that can move to C-Mode but that doesn't mean NetApp should force everyone to C-Mode. We were actually going to go to it on a new cluster earlier this year and then were advised against it due to a bug (which has been fixed).
We'll probably consider it next year sometime. Just not for existing infrastructure. C-
Bohemen, Anton van wrote:
Well, to be honest, I think it is time to (at least) start thinking about moving to cluster mode. A FAS2220 is a bit on the positive side as you can’t do SAN with it in C-Mode, but a 2240 can, so then you are fine. I’ve been testing quite extensive with the simulator and it actually works fine. A bit to my surprise as well I must admit, I was just as skeptical as you all. The concepts are great and the VServer virtualization layer is a great addition to the WAFL storage virtualization layer. VServers work so much better than vFilers and with c-mode you get all the scalability advantages you need.
So, I’m not surprised all the commercial people at NetApp and partners are pushing to C-Mode. If you have a technology refresh in a few years, you should definitely consider it. If you have one this year, I’m not really sure yet. Not because not all features are supported in c-mode (yet), but right now it is quite a steep learning curve if you’re used to 7-mode. You need to re-think most of your storage setup but once you do, I think c-mode will fit a lot of environments.
Bottom line: start reading and testing, I think c-mode actually is a good product.
Regards,
Anton
*From:* toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] *On Behalf Of *Blackmor, Chris *Sent:* zondag 17 juni 2012 17:45 *To:* Milazzo Giacomo *Cc:* toasters@teaparty.net *Subject:* Re: Cluster mode - Market vs field
They can say that all they want but I think NA has had way too many missteps with C-mode to be pushing it down anyone's
Sent from my iPhone
On Jun 17, 2012, at 10:24 AM, "Milazzo Giacomo" <G.Milazzo@sinergy.it mailto:G.Milazzo@sinergy.it> wrote:
Hi all, I’ve recently attended an official event where people from NetApp in few words said: “/DOT 8 7-mode has done its time, it’s over, it has been a great os but It’s now time to move and push DOT 8 Cluster Mode, everytime, everywhere…/” and this also if with a simple two node HA system, including 2220! What do you think about? Also let us assume that cluster mode can (I add ‘theoretically’) do all what 7 mode does, don’t you think that is quite premature to trumpet this out? Regards, _______________________________________________ Toasters mailing list Toasters@teaparty.net <mailto:Toasters@teaparty.net> http://www.teaparty.net/mailman/listinfo/toastersThis e-mail is personal. For our full disclaimer, please visit www.centric.eu/disclaimer.
Hi,
While I wouldn’t disagree that it’s a good product, having run GX, then 8.0C then 8.1C I’d argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
In fact, I’d probably wager that there will be a minimal performance degradation too. Less speed and reliability is a hard sell and that’s before you even think about how difficult it’d be to move to C mode without buying more hardware.
Darren
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Bohemen, Anton van Sent: 18 June 2012 10:56 To: toasters@teaparty.net Subject: RE: Cluster mode - Market vs field
Well, to be honest, I think it is time to (at least) start thinking about moving to cluster mode. A FAS2220 is a bit on the positive side as you can’t do SAN with it in C-Mode, but a 2240 can, so then you are fine. I’ve been testing quite extensive with the simulator and it actually works fine. A bit to my surprise as well I must admit, I was just as skeptical as you all. The concepts are great and the VServer virtualization layer is a great addition to the WAFL storage virtualization layer. VServers work so much better than vFilers and with c-mode you get all the scalability advantages you need.
So, I’m not surprised all the commercial people at NetApp and partners are pushing to C-Mode. If you have a technology refresh in a few years, you should definitely consider it. If you have one this year, I’m not really sure yet. Not because not all features are supported in c-mode (yet), but right now it is quite a steep learning curve if you’re used to 7-mode. You need to re-think most of your storage setup but once you do, I think c-mode will fit a lot of environments.
Bottom line: start reading and testing, I think c-mode actually is a good product.
Regards,
Anton
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blackmor, Chris Sent: zondag 17 juni 2012 17:45 To: Milazzo Giacomo Cc: toasters@teaparty.net Subject: Re: Cluster mode - Market vs field
They can say that all they want but I think NA has had way too many missteps with C-mode to be pushing it down anyone's
Sent from my iPhone
On Jun 17, 2012, at 10:24 AM, "Milazzo Giacomo" <G.Milazzo@sinergy.itmailto:G.Milazzo@sinergy.it> wrote: Hi all,
I’ve recently attended an official event where people from NetApp in few words said:
“DOT 8 7-mode has done its time, it’s over, it has been a great os but It’s now time to move and push DOT 8 Cluster Mode, everytime, everywhere…” and this also if with a simple two node HA system, including 2220!
What do you think about? Also let us assume that cluster mode can (I add ‘theoretically’) do all what 7 mode does, don’t you think that is quite premature to trumpet this out?
Regards,
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click herehttps://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg==.
This e-mail is personal. For our full disclaimer, please visit www.centric.eu/disclaimer.
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel within a 3 weeks or a month I was pretty comfortable with it. - there is a back way into the old ontap cli (it's not exactly the same, but close) for some debugging things. - the latency isn't nearly as bad as you might think. We had Nexus 5k's and many 10g links, and worked pretty smoothly. - While stability wasn't 100%, the failovers worked as expected and filesystems never stopped being served. This was 8.0 C-Mode - Moving data between cfo pairs is slick and awesome. Highly recommend 64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I was trying to be diplomatic, seems Michael has lowered the bar on that count :)
I agree with you both. I was very concerned about the latency when accessing remote dblades in our environment since we have lots of very small files where the difference is highlighted. There was supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that.
At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move.
I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond?
We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access remote nodes. Not that they don't have other performance gotchas.
Darren
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blake Golliher Sent: 19 June 2012 17:43 To: Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel within a 3 weeks or a month I was pretty comfortable with it. - there is a back way into the old ontap cli (it's not exactly the same, but close) for some debugging things. - the latency isn't nearly as bad as you might think. We had Nexus 5k's and many 10g links, and worked pretty smoothly. - While stability wasn't 100%, the failovers worked as expected and filesystems never stopped being served. This was 8.0 C-Mode - Moving data between cfo pairs is slick and awesome. Highly recommend 64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote" access would never happen when LIFs are configured on each node (which is the recommended best practice configuration). The only case when you'd get remote traffic thru the interconnect 10GigE network is when you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the destination node and that would provide a direct path to the new destination node for the volume; one LIF on each node is the bottom line recommended best practice).
Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode: http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum mary.pdf Regarding the new CLI, I really like it. Conceptually, it's similar to Cisco's CLI (contextual commands, contextual help, tab to complete the command or get help). The tab and help features are really great.
I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out. Perhaps, download the cluster-mode vSim first and try it on your laptop. ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system anymore. There's a lot of goodness to it now and it's stable and really easy to work with. You could check the Data ONTAP Cluster-Mode Administration course begin taught by NetApp training partners to test drive it further if you wish.
Cheers, Sil Angelescu
On 6/19/12 12:57 PM, "Darren Sykes" Darren.Sykes@csr.com wrote:
I was trying to be diplomatic, seems Michael has lowered the bar on that count :)
I agree with you both. I was very concerned about the latency when accessing remote dblades in our environment since we have lots of very small files where the difference is highlighted. There was supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that.
At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move.
I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond?
We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access remote nodes. Not that they don't have other performance gotchas.
Darren
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blake Golliher Sent: 19 June 2012 17:43 To: Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel
within a 3 weeks or a month I was pretty comfortable with it.
- there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think. We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served. This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome. Highly
recommend 64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Angelescu, Silviu (NetApp) wrote:
Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote" access would never happen when LIFs are configured on each node (which is the recommended best practice configuration).
That someone was me. If you consider an NFSv3 based (not NFSv4 with referrals or v4.1 with pNFS extension) environment with many 100s of NFS clients and they have > 100 NFS mounts each to various places in the GNS from the C-mode cluster, and they cannot really ever be restarted all at once except under exceptional circumstances (meaning: re-mount happens very rarely for a large portion of the data...)
Even if LIFs are configured on each node, as soon as you move things around in the back-end, you'll get lots of cross Cluster Network accesses. It's pretty much unavoidable the way I see things
The only case when you'd get remote traffic thru the interconnect 10GigE network is when you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the destination node and that would provide a direct path to the new destination node for the volume; one LIF on each node is the bottom line recommended best practice).
That's fine but in the above scenario i described it won't really help that much
Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode: http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum mary.pdf
It's a nice result, comparable to any other major "Tier 1" Block Storage vendor out there
The latency impact of C-mode compared to 7-mode cannot be juged from that data though, simply because there's no published SPC-1 benchmark for a 7-mode "single node" (or HA cluster) FAS6240. If there were, then I would absolutely do that comparison for FC Block Storage use case as well, not just SPECsfs2008
Regards, /M
I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade without pnfs.
The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster network most of the time.
There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.
Darren Sent from my iPhone
On 19 Jun 2012, at 18:33, "Angelescu, Silviu" Silviu.Angelescu@netapp.com wrote:
Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote" access would never happen when LIFs are configured on each node (which is the recommended best practice configuration). The only case when you'd get remote traffic thru the interconnect 10GigE network is when you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the destination node and that would provide a direct path to the new destination node for the volume; one LIF on each node is the bottom line recommended best practice).
Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode: http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum mary.pdf Regarding the new CLI, I really like it. Conceptually, it's similar to Cisco's CLI (contextual commands, contextual help, tab to complete the command or get help). The tab and help features are really great.
I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out. Perhaps, download the cluster-mode vSim first and try it on your laptop. ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system anymore. There's a lot of goodness to it now and it's stable and really easy to work with. You could check the Data ONTAP Cluster-Mode Administration course begin taught by NetApp training partners to test drive it further if you wish.
Cheers, Sil Angelescu
On 6/19/12 12:57 PM, "Darren Sykes" Darren.Sykes@csr.com wrote:
I was trying to be diplomatic, seems Michael has lowered the bar on that count :)
I agree with you both. I was very concerned about the latency when accessing remote dblades in our environment since we have lots of very small files where the difference is highlighted. There was supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that.
At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move.
I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond?
We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access remote nodes. Not that they don't have other performance gotchas.
Darren
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blake Golliher Sent: 19 June 2012 17:43 To: Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel
within a 3 weeks or a month I was pretty comfortable with it.
- there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think. We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served. This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome. Highly
recommend 64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
With the access methods available today (either RRDNS or the onboard load balancing) a client can land on any lif regardless of the intended access target. pNFS fixes that, but there's no telling how far away that really is.
We see more traffic on the intercluster network than on the data network, regularly. Granted, we move volumes all the time (being able to load balance and expand capacity at will without 7 mode restraints is fantastic) and as we add nodes that traffic increases, but even without that there is and will be a significant amount of cluster traffic that is due to non-local lif's.
If your need is truly very high performance, cluster mode is not for you, at least not now. For marginally high performance and lower it performs well enough. If you are able/willing to trade some of the features of 7 mode (ok, let's be honest, it's quite a few features still) for the primary benefits of CM, you are unlikely to be disappointed. For the data set we put on CM we can do without those features, for now, but the benefits of the namespace and transparent volume migrations for capacity or load balancing is a major benefit. Enough to outweigh all the features we've yet to see on CM.
Jeff Kennedy Qualcomm, Incorporated QCT Engineering Compute 858-651-6592
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Darren Sykes Sent: Tuesday, June 19, 2012 12:57 PM To: Angelescu, Silviu Cc: Michael Bergman; Toasters Subject: Re: Cluster mode - Market vs field
I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade without pnfs.
The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster network most of the time.
There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.
Darren Sent from my iPhone
On 19 Jun 2012, at 18:33, "Angelescu, Silviu" Silviu.Angelescu@netapp.com wrote:
Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote" access would never happen when LIFs are configured on each node (which is the recommended best practice configuration). The only case when you'd get remote traffic thru the interconnect 10GigE network is when you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the destination node and that would provide a direct path to the new destination node for the volume; one LIF on each node is the bottom line recommended best practice).
Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode: http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp /A001 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executiv e-sum mary.pdf Regarding the new CLI, I really like it. Conceptually, it's similar to Cisco's CLI (contextual commands, contextual help, tab to complete the command or get help). The tab and help features are really great.
I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out. Perhaps, download the cluster-mode vSim first and try it on your laptop. ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system anymore. There's a lot of goodness to it now and it's stable and really easy to work with. You could check the Data ONTAP Cluster-Mode Administration course begin taught by NetApp training partners to test drive it further if you wish.
Cheers, Sil Angelescu
On 6/19/12 12:57 PM, "Darren Sykes" Darren.Sykes@csr.com wrote:
I was trying to be diplomatic, seems Michael has lowered the bar on that count :)
I agree with you both. I was very concerned about the latency when accessing remote dblades in our environment since we have lots of very small files where the difference is highlighted. There was supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that.
At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move.
I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond?
We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access remote nodes. Not that they don't have other performance gotchas.
Darren
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blake Golliher Sent: 19 June 2012 17:43 To: Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel
within a 3 weeks or a month I was pretty comfortable with it.
- there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think. We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served. This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome. Highly
recommend 64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I'm really glad that my questions started a so interesting discussion. And of course I'm also glad to read that speaking about "cluster mode" has to be based on a minimum of a two nodes (four heads) because it could be a really nonsense to speak about GNS, grid storage, HPC and so on in a "small" environment where budget and needs are no more that a 2220 with 10k SAS disks... Isilon (that I know) has also be cited as the really competitor to compare with - and the same opinion about NetApp comes from Isilon, part of but not from EMC of course :-). But just because I know a lot of things about Isilon technology (from field) I still continue to think that disk ownership can be the real constraint from a pure performance (in HPC word) point of view.
Thank you everybody
-----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Kennedy, Jeffrey Inviato: mercoledì 20 giugno 2012 00:47 A: Darren Sykes; Angelescu, Silviu Cc: Michael Bergman; Toasters Oggetto: RE: Cluster mode - Market vs field
With the access methods available today (either RRDNS or the onboard load balancing) a client can land on any lif regardless of the intended access target. pNFS fixes that, but there's no telling how far away that really is.
We see more traffic on the intercluster network than on the data network, regularly. Granted, we move volumes all the time (being able to load balance and expand capacity at will without 7 mode restraints is fantastic) and as we add nodes that traffic increases, but even without that there is and will be a significant amount of cluster traffic that is due to non-local lif's.
If your need is truly very high performance, cluster mode is not for you, at least not now. For marginally high performance and lower it performs well enough. If you are able/willing to trade some of the features of 7 mode (ok, let's be honest, it's quite a few features still) for the primary benefits of CM, you are unlikely to be disappointed. For the data set we put on CM we can do without those features, for now, but the benefits of the namespace and transparent volume migrations for capacity or load balancing is a major benefit. Enough to outweigh all the features we've yet to see on CM.
Jeff Kennedy Qualcomm, Incorporated QCT Engineering Compute 858-651-6592
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Darren Sykes Sent: Tuesday, June 19, 2012 12:57 PM To: Angelescu, Silviu Cc: Michael Bergman; Toasters Subject: Re: Cluster mode - Market vs field
I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade without pnfs.
The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster network most of the time.
There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.
Darren Sent from my iPhone
On 19 Jun 2012, at 18:33, "Angelescu, Silviu" Silviu.Angelescu@netapp.com wrote:
Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote" access would never happen when LIFs are configured on each node (which is the recommended best practice configuration). The only case when you'd get remote traffic thru the interconnect 10GigE network is when you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the destination node and that would provide a direct path to the new destination node for the volume; one LIF on each node is the bottom line recommended best practice).
Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode: http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp /A001 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executiv e-sum mary.pdf Regarding the new CLI, I really like it. Conceptually, it's similar to Cisco's CLI (contextual commands, contextual help, tab to complete the command or get help). The tab and help features are really great.
I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out. Perhaps, download the cluster-mode vSim first and try it on your laptop. ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system anymore. There's a lot of goodness to it now and it's stable and really easy to work with. You could check the Data ONTAP Cluster-Mode Administration course begin taught by NetApp training partners to test drive it further if you wish.
Cheers, Sil Angelescu
On 6/19/12 12:57 PM, "Darren Sykes" Darren.Sykes@csr.com wrote:
I was trying to be diplomatic, seems Michael has lowered the bar on that count :)
I agree with you both. I was very concerned about the latency when accessing remote dblades in our environment since we have lots of very small files where the difference is highlighted. There was supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that.
At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move.
I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond?
We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access remote nodes. Not that they don't have other performance gotchas.
Darren
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blake Golliher Sent: 19 June 2012 17:43 To: Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel
within a 3 weeks or a month I was pretty comfortable with it.
- there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think. We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served. This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome. Highly
recommend 64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I think that's the Netapp argument - regardless of the number of nodes, C mode is the future.
I'm also not sure I understand your terminology. A cluster is either one or an even number of nodes. A node is effectively a filer or a head (a node certainly isn't two heads/filers).
Isilon is slightly different - the minimum cluster size is 3 nodes but you can have an odd number of nodes and add to the cluster in singles. Isilon also doesn't have the concept of disk ownership as each node has its own disks and doesn't share with any others. Resilience is achieved by either mirroring or adding parity information for all data that is written.
Netapp on the other hand will have the problem of disk ownership because data isn't replicated - resilience is gained from using RAID within a pair of nodes.
Darren
-----Original Message----- From: Milazzo Giacomo [mailto:G.Milazzo@sinergy.it] Sent: 20 June 2012 13:23 To: Kennedy, Jeffrey; Darren Sykes; Angelescu, Silviu Cc: Michael Bergman; Toasters Subject: R: Cluster mode - Market vs field
I'm really glad that my questions started a so interesting discussion. And of course I'm also glad to read that speaking about "cluster mode" has to be based on a minimum of a two nodes (four heads) because it could be a really nonsense to speak about GNS, grid storage, HPC and so on in a "small" environment where budget and needs are no more that a 2220 with 10k SAS disks... Isilon (that I know) has also be cited as the really competitor to compare with - and the same opinion about NetApp comes from Isilon, part of but not from EMC of course :-). But just because I know a lot of things about Isilon technology (from field) I still continue to think that disk ownership can be the real constraint from a pure performance (in HPC word) point of view.
Thank you everybody
-----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Kennedy, Jeffrey Inviato: mercoledì 20 giugno 2012 00:47 A: Darren Sykes; Angelescu, Silviu Cc: Michael Bergman; Toasters Oggetto: RE: Cluster mode - Market vs field
With the access methods available today (either RRDNS or the onboard load balancing) a client can land on any lif regardless of the intended access target. pNFS fixes that, but there's no telling how far away that really is.
We see more traffic on the intercluster network than on the data network, regularly. Granted, we move volumes all the time (being able to load balance and expand capacity at will without 7 mode restraints is fantastic) and as we add nodes that traffic increases, but even without that there is and will be a significant amount of cluster traffic that is due to non-local lif's.
If your need is truly very high performance, cluster mode is not for you, at least not now. For marginally high performance and lower it performs well enough. If you are able/willing to trade some of the features of 7 mode (ok, let's be honest, it's quite a few features still) for the primary benefits of CM, you are unlikely to be disappointed. For the data set we put on CM we can do without those features, for now, but the benefits of the namespace and transparent volume migrations for capacity or load balancing is a major benefit. Enough to outweigh all the features we've yet to see on CM.
Jeff Kennedy Qualcomm, Incorporated QCT Engineering Compute 858-651-6592
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Darren Sykes Sent: Tuesday, June 19, 2012 12:57 PM To: Angelescu, Silviu Cc: Michael Bergman; Toasters Subject: Re: Cluster mode - Market vs field
I'll back up what has already been said - there's absolutely no guarantee you'll access a local dblade without pnfs.
The other thing to consider is infinite volumes - with no nfs4 support, by nature you'll use the cluster network most of the time.
There's no need for a sim - we're running 8.1p1 in our live environment, and will run 8.1.1RC in dev.
Darren Sent from my iPhone
On 19 Jun 2012, at 18:33, "Angelescu, Silviu" Silviu.Angelescu@netapp.com wrote:
Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote" access would never happen when LIFs are configured on each node (which is the recommended best practice configuration). The only case when you'd get remote traffic thru the interconnect 10GigE network is when you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the destination node and that would provide a direct path to the new destination node for the volume; one LIF on each node is the bottom line recommended best practice).
Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode: http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp /A001 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executiv e-sum mary.pdf Regarding the new CLI, I really like it. Conceptually, it's similar to Cisco's CLI (contextual commands, contextual help, tab to complete the command or get help). The tab and help features are really great.
I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out. Perhaps, download the cluster-mode vSim first and try it on your laptop. ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system anymore. There's a lot of goodness to it now and it's stable and really easy to work with. You could check the Data ONTAP Cluster-Mode Administration course begin taught by NetApp training partners to test drive it further if you wish.
Cheers, Sil Angelescu
On 6/19/12 12:57 PM, "Darren Sykes" Darren.Sykes@csr.com wrote:
I was trying to be diplomatic, seems Michael has lowered the bar on that count :)
I agree with you both. I was very concerned about the latency when accessing remote dblades in our environment since we have lots of very small files where the difference is highlighted. There was supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that.
At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move.
I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond?
We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access remote nodes. Not that they don't have other performance gotchas.
Darren
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blake Golliher Sent: 19 June 2012 17:43 To: Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel
within a 3 weeks or a month I was pretty comfortable with it.
- there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think. We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served. This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome. Highly
recommend 64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Pardon the interjection, but I thought this may be of interest to those of you using 7 and looking to get a quick introduction to Cluster-Mode.
"The new Introduction to Cluster-Mode for 7-Mode Administrators course provides an introduction to Cluster-Mode, and is being offered free of charge. This 1 hour Web-based training is a great way to introduce 7-Mode familiar NetApp customers, employees and partners to the basics of Cluster-Mode, an important Data ONTAP concept and key NetApp differentiator."
Course Description: http://learningcenter.netapp.com/content/public/production/course_descriptio... Enrollment: http://learningcenter.netapp.com/LC?ObjectType=WBT&ObjectID=00236611 (NOW login required)
Alex McDonald NetApp, CTO Office Industry Stds and Forums twitter: @alextangent phone: +44 7795 046686
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Angelescu, Silviu Sent: Tuesday, June 19, 2012 6:34 PM To: Darren Sykes; Blake Golliher; Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
Someone mentioned below "With C-mode there's "local" and "remote" array access". Actually, "remote" access would never happen when LIFs are configured on each node (which is the recommended best practice configuration). The only case when you'd get remote traffic thru the interconnect 10GigE network is when you need to shutdown a node for an upgrade for example, or when you move a volume to a node where you don't have a LIF configured (but you could also move the LIF along with the volume or just create a new LIF on the destination node and that would provide a direct path to the new destination node for the volume; one LIF on each node is the bottom line recommended best practice).
Regarding the "performance impact" of cluster-mode, you may want to check the latest SPC-1 benchmark results with FAS 6240 in cluster-mode: http://www.storageperformance.org/benchmark_results_files/SPC-1/NetApp/A001 15_NetApp_FAS6240-cluster/a00115_NetApp_FAS6240-cluster_SPC-1_executive-sum mary.pdf Regarding the new CLI, I really like it. Conceptually, it's similar to Cisco's CLI (contextual commands, contextual help, tab to complete the command or get help). The tab and help features are really great.
I suggest looking up the data about ONTAP 8.1 cluster-mode and try it out. Perhaps, download the cluster-mode vSim first and try it on your laptop. ONTAP 8.1 cluster-mode is not my "grandpa's ONTAP cluster-mode" system anymore. There's a lot of goodness to it now and it's stable and really easy to work with. You could check the Data ONTAP Cluster-Mode Administration course begin taught by NetApp training partners to test drive it further if you wish.
Cheers, Sil Angelescu
On 6/19/12 12:57 PM, "Darren Sykes" Darren.Sykes@csr.com wrote:
I was trying to be diplomatic, seems Michael has lowered the bar on that count :)
I agree with you both. I was very concerned about the latency when accessing remote dblades in our environment since we have lots of very small files where the difference is highlighted. There was supposed to be significant improvements in 8.1 - I should really re-run the tests to prove that.
At the moment we attempt to work around the problem by tracking where volumes live and mounting the nblade local to the dblade. Admittedly, that's not the most elegant solution and there is a lag as volumes move.
I wonder how that latency will impact the performance of infinite volumes in 8.1C and beyond?
We also have Isilons and their infiniband backend proved extremely good in reducing the overhead of access remote nodes. Not that they don't have other performance gotchas.
Darren
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Blake Golliher Sent: 19 June 2012 17:43 To: Michael Bergman Cc: Toasters Subject: Re: Cluster mode - Market vs field
In fairness, I used C-Mode at my last place and I can say a few things.
- the new cli isn't has hard to pick up as you might think. I feel
within a 3 weeks or a month I was pretty comfortable with it.
- there is a back way into the old ontap cli (it's not exactly the
same, but close) for some debugging things.
- the latency isn't nearly as bad as you might think. We had Nexus
5k's and many 10g links, and worked pretty smoothly.
- While stability wasn't 100%, the failovers worked as expected and
filesystems never stopped being served. This was 8.0 C-Mode
- Moving data between cfo pairs is slick and awesome. Highly recommend
64bit aggrs.
All that said, I don't think a C-Mode cluster should be pushed unless 4 nodes are in it, two cfo pairs is what every c-mode cluster should start as. Two nodes doesn't make a heck of a lot of sense. I now live in a sea of Isilon and comparing and contrasting the two platforms is pretty interesting.
Hope that helps,
-Blake
On Tue, Jun 19, 2012 at 8:40 AM, Michael Bergman michael.bergman@ericsson.com wrote:
Darren Sykes wrote:
While I wouldn't disagree that it's a good product, having run GX, then 8.0C then 8.1C I'd argue that in a 2 node configuration C mode would be less reliable with little benefit over 7 mode at this point in time.
Quite... Functionality wise, C-Mode isn't finished yet either, there's quite some work for NTAP to do still. And it's not stable in the proper sense of the word -- 8.1 7-mode isn't stable yet either.
In fact, I'd probably wager that there will be a minimal performance degradation too.
There is. But it's not minimal, not in any sense of the word. With C-mode there's "local" and "remote" array access (the back-ends below the heads so to speak) and the additional latency induced by the cluster network is big. Really big. In use cases where you need to be down at 1 ms avg internal latency for NFS (with the help of PAM-II etc), for a fair NFSops load point, you can just forget about C-Mode yet. *Maybe* some time in the future, maybe it can be polished until 8.2Px or 8.2.1 is out... We'll see.
Less speed and reliability is a hard sell and that's before you even think about how difficult it'd be to move to C mode without buying more hardware.
Somehow "moving" to C-Mode without deploying a whole new system from scratch is unthinkable from there I stand. But of course it depends on the use case and other details.
I'd argue that at this point at least and for the next 24 months C-mode is pretty much like deploying a completely new (from a different vendor) storage system at your site in your environment, effectively potentiually doubling the complexity to handle for your storage ops ppl. Remember the CLI is completely different as well (for the large group of us who need/want to use that a lot, suppose a no of customers use the Web GUI way of controlling Filers to 100%)
/M _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To report this email as spam click https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg== .
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I really appreciate this contribution. Thanks,
Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Bohemen, Anton van Inviato: lunedì 18 giugno 2012 11:56 A: toasters@teaparty.net Oggetto: RE: Cluster mode - Market vs field
Well, to be honest, I think it is time to (at least) start thinking about moving to cluster mode. A FAS2220 is a bit on the positive side as you can’t do SAN with it in C-Mode, but a 2240 can, so then you are fine. I’ve been testing quite extensive with the simulator and it actually works fine. A bit to my surprise as well I must admit, I was just as skeptical as you all. The concepts are great and the VServer virtualization layer is a great addition to the WAFL storage virtualization layer. VServers work so much better than vFilers and with c-mode you get all the scalability advantages you need.
So, I’m not surprised all the commercial people at NetApp and partners are pushing to C-Mode. If you have a technology refresh in a few years, you should definitely consider it. If you have one this year, I’m not really sure yet. Not because not all features are supported in c-mode (yet), but right now it is quite a steep learning curve if you’re used to 7-mode. You need to re-think most of your storage setup but once you do, I think c-mode will fit a lot of environments.
Bottom line: start reading and testing, I think c-mode actually is a good product.
Regards,
Anton
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net]mailto:[mailto:toasters-bounces@teaparty.net] On Behalf Of Blackmor, Chris Sent: zondag 17 juni 2012 17:45 To: Milazzo Giacomo Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: Cluster mode - Market vs field
They can say that all they want but I think NA has had way too many missteps with C-mode to be pushing it down anyone's
Sent from my iPhone
On Jun 17, 2012, at 10:24 AM, "Milazzo Giacomo" <G.Milazzo@sinergy.itmailto:G.Milazzo@sinergy.it> wrote: Hi all,
I’ve recently attended an official event where people from NetApp in few words said:
“DOT 8 7-mode has done its time, it’s over, it has been a great os but It’s now time to move and push DOT 8 Cluster Mode, everytime, everywhere…” and this also if with a simple two node HA system, including 2220!
What do you think about? Also let us assume that cluster mode can (I add ‘theoretically’) do all what 7 mode does, don’t you think that is quite premature to trumpet this out?
Regards,
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
This e-mail is personal. For our full disclaimer, please visit www.centric.eu/disclaimerhttp://www.centric.eu/disclaimer.
With all the missteps NA has had getting C-mode out the door I don't think they can really push it on anyone. Remember Auspex 1.8. Where's Auspex now?
I know they have fixed many of the early C-mode issues but I also know we are going to approach it very carefully when it has the full feature set. C-
Sent from my iPhone
On Jun 17, 2012, at 10:24 AM, "Milazzo Giacomo" <G.Milazzo@sinergy.itmailto:G.Milazzo@sinergy.it> wrote:
Hi all,
I’ve recently attended an official event where people from NetApp in few words said:
“DOT 8 7-mode has done its time, it’s over, it has been a great os but It’s now time to move and push DOT 8 Cluster Mode, everytime, everywhere…” and this also if with a simple two node HA system, including 2220!
What do you think about? Also let us assume that cluster mode can (I add ‘theoretically’) do all what 7 mode does, don’t you think that is quite premature to trumpet this out?
Regards,
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
On Jun 17, 2012 12:08 PM, "Blackmor, Chris" Chris.Blackmor@amd.com wrote:
With all the missteps NA has had getting C-mode out the door I don't think they can really push it on anyone. Remember Auspex 1.8. Where's Auspex now?
A very fond memory, if triste; the months that felt like years that I spent trying to master the art of handholding the recovery of lock state after failover, and appreciating the cleverness of the multicast tricks and nfs protocol.shortcuts, taught me lessons about how NFS really works (when it does) that will last me.
H-A is great when it works, but I've rarely seen implementations that aren't seriously lower availability than the single-point-of-failure system they try to replace.
Though I think what did for Auspex wasn't their ambitious H-A, it was NetApp's ability to cling tenatiously to the Juggernaut as Moore's Law drove costs of processing, memory, and bandwidth down, and down, and ever down.
I agree with you and Todd but I've another point of view. I know very well other competitor (really ones) in the field of 'cluster' such as Isilon (and this one considers a real competitor only NetApp). For Isilon and such companies cluster mean over all HPC, parallel f/s, single name spaces, grid computing, they mean also to share network and other resources in a scale-out point of view.
I think for that reason that to propose CM to justify a 'never poweroff', 'move a single vol without interruption' and so on is a very limited vision maybe because in the scale-out sharing of resource the NetApp disk ownership is a constraint? Or more realistically because to speak of CM thinking to 2200 systems could be a nonsense (customer with limited budget and needs..
regards
-----Messaggio originale----- Da: Bennett Todd [mailto:bent@latency.net] Per conto di Bennett Todd Inviato: domenica 17 giugno 2012 18:27 A: Blackmor, Chris; Milazzo Giacomo Cc: toasters@teaparty.net Oggetto: Re: Cluster mode - Market vs field
On Jun 17, 2012 12:08 PM, "Blackmor, Chris" Chris.Blackmor@amd.com wrote:
With all the missteps NA has had getting C-mode out the door I don't think they can really push it on anyone. Remember Auspex 1.8. Where's Auspex now?
A very fond memory, if triste; the months that felt like years that I spent trying to master the art of handholding the recovery of lock state after failover, and appreciating the cleverness of the multicast tricks and nfs protocol.shortcuts, taught me lessons about how NFS really works (when it does) that will last me.
H-A is great when it works, but I've rarely seen implementations that aren't seriously lower availability than the single-point-of-failure system they try to replace.
Though I think what did for Auspex wasn't their ambitious H-A, it was NetApp's ability to cling tenatiously to the Juggernaut as Moore's Law drove costs of processing, memory, and bandwidth down, and down, and ever down.
-
Angelescu, Silviu -
Bennett Todd -
Blackmor, Chris -
Blackmor, Chris -
Blake Golliher -
Bohemen, Anton van -
Darren Sykes -
Kennedy, Jeffrey -
McDonald, Alex -
Michael Bergman -
Milazzo Giacomo