Jeremy,

If you consider that the storage industry as having 5 main providers, IBM, EMC, HP, HDS, & NetApp, you will note that only NetApp provides NFS on their primary storage arrays. I did not list IBM N-Series (NetApp OEM) arrays as they are not IBM's primary array (which is the Shark). In addition, NFS has only been an option in the VMware space for ~18 months. So to begin NFS has a much smaller footprint due to vendor offerings and historical choices with implementation. With that said it has a lot of momentum.

On the surface the NFS (vs FCP) performance concerns seem logical. What we finding is that with virtualization, and more specifically the components of, these aren't real issues.

Does FCP outperform NFS? Yes – when the measurement is how fast can 1 server access one area of storage Yes – when the measurement is how fast can 1 virtual machine access on Datastore No – when the measurement is how much IO can an individual Datastore push to multiple VMs on multiple ESX hosts. This is why compute grids are NFS and not FCP

Does NFS consume more CPU than FCP? Yes – but our IO benchmarks show that number is ~ 20%, or in other words, CPU are powerful and abundant today, so its not an issue.

Is NFS less expensive than FCP? Yes – no switches, optical cables, HBAs

Does NFS provide greater storage utilization? Yes – VMDKs are thin provisioned Yes, - No dead space residing in VMFS file systems. NFS removes one layer in the storage stack, allowing VMware to treat disks as files, (which is what VMFS does).

In additon NFS significantly simplifies many operational tasks. Consider the requirements to grow a Datastore. On FCP you create a LUN, map it, scan for it on ESX, add a VMFS extent, and then scan for the new LUN and extent on all remaining ESX servers. With NFS on NetApp its one command and you're done, oh and I forgot, you can also shrink NFS just as easily. Try that with any SAN file system.

Our founder, Dave Hitz, went to Berkley with Mendel Rosenblum, the founder of VMware. As a grad student Mendel worked on a file system project named LFS. Dave, worked on ideas for WAFL while at Berkley and learned from the LFS project. The two met last week, and Dave asked Mendel his thoughts on NFS, to which Mendel replied, 'NetApp will always take storage virtualization to places where we will never go' and ' VMware needed VMFS in order to treat storage as files thus enabling features like VMotion.' Some of this was posted on Dave's blog: http://blogs.netapp.com/dave/

So, I would ask if NFS performs on par with FCP in a scaled out model, than doesn't simplicity and higher utilization rule in favor of NFS? My intention is not to start a protocol war of FCP vs NFS rather I am merely asking the question...

Vaughn Stewart Virtualization Evangelist NetApp

P.S. try a benchmark like I described...

On 9/27/07, Page, Jeremy jeremy.page@gilbarco.com wrote:

Thanks, I read Dave's blog and the link to the SE that was at the bottom. Some good information there, but no hard numbers. I'd really like to talk to someone that has both in production or maybe done testing on the same box with iSCSI and NFS, mainly because I'm too lazy to do it properly myself.

---

*From:* Doucet, Hugo [mailto:hdoucet@netapp.com] *Sent:* Thursday, September 27, 2007 10:37 AM *To:* Page, Jeremy; toasters@mathworks.com *Subject:* Re: NFS and VMware

Jeremy

I suggest you consult this weeks posts on Dave Hitz blog (NetApp founder) and people's comments.

The blog can be found via a link right pane on www.netapp.com

Hugo (NetApp employee)

-----Original Message----- From: Page, Jeremy jeremy.page@gilbarco.com To: toasters@mathworks.com toasters@mathworks.com Sent: Thu Sep 27 14:26:47 2007 Subject: NFS and VMware

Are any of the toaster folks using NFS for VMware in a production environment? We have about 150 VMs across 12 servers at the moment sitting on an IBM DS4500, but I just got my budget approved to buy a nice 3040 cluster. For ease of backup/restore/DR I really would like to use NFS instead of FCP/iSCSI but I don't know what the performance impact will be. Any comments/experiences?

This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.

This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.

With Oracle bundling an NFS client in their database release there's a statement in itself.

-----Original Message----- From: owner-toasters@mathworks.com on behalf of Mcdonald, Alex Sent: Thu 9/27/2007 8:07 AM To: Doucet, Hugo; jeremy.page@gilbarco.com; toasters@mathworks.com Subject: RE: NFS and VMware

Jeremy;

http://storagefoo.blogspot.com/2007_09_09_archive.html http://storagefoo.blogspot.com/2007_09_09_archive.html (backup & recovery stuff under NFS) and http://storagefoo.blogspot.com/2007/09/vmware-over-nfs.html http://storagefoo.blogspot.com/2007/09/vmware-over-nfs.html , both from Nick Triantos' blog (a NetApp employee). Quote: "People may find this hard to believe, but the performance over NFS is actually better than FC or iSCSI not only in terms of throughtput but also in terms of latency."

Plus he has another entry http://storagefoo.blogspot.com/2007/09/demos-from-vmworld.html that points at a VMware over NFS demo from VMworld.

alex mcdonald | netapp

vm: +44 7795 046686 | em: alexmc@netapp.com |

________________________________

From: Doucet, Hugo Sent: 27 September 2007 15:37 To: jeremy.page@gilbarco.com; toasters@mathworks.com Subject: Re: NFS and VMware

Jeremy

I suggest you consult this weeks posts on Dave Hitz blog (NetApp founder) and people's comments.

The blog can be found via a link right pane on www.netapp.com

Hugo (NetApp employee)

-----Original Message----- From: Page, Jeremy jeremy.page@gilbarco.com To: toasters@mathworks.com toasters@mathworks.com Sent: Thu Sep 27 14:26:47 2007 Subject: NFS and VMware

Are any of the toaster folks using NFS for VMware in a production environment? We have about 150 VMs across 12 servers at the moment sitting on an IBM DS4500, but I just got my budget approved to buy a nice 3040 cluster. For ease of backup/restore/DR I really would like to use NFS instead of FCP/iSCSI but I don't know what the performance impact will be. Any comments/experiences?

This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.

Who do you think provides Nick his info?

V

On 9/27/07, Mcdonald, Alex Alex.Mcdonald@netapp.com wrote:

Jeremy;

http://storagefoo.blogspot.com/2007_09_09_archive.html (backup & recovery stuff under NFS) and http://storagefoo.blogspot.com/2007/09/vmware-over-nfs.html, both from Nick Triantos' blog (a NetApp employee). Quote: "People may find this hard to believe, but the performance over NFS is actually better than FC or iSCSI not only in terms of throughtput but also in terms of latency."

Plus he has another entry http://storagefoo.blogspot.com/2007/09/demos-from-vmworld.html that points at a VMware over NFS demo from VMworld.

alex mcdonald | netapp

vm: +44 7795 046686 | em: alexmc@netapp.com |

---

*From:* Doucet, Hugo *Sent:* 27 September 2007 15:37 *To:* jeremy.page@gilbarco.com; toasters@mathworks.com *Subject:* Re: NFS and VMware

Jeremy

I suggest you consult this weeks posts on Dave Hitz blog (NetApp founder) and people's comments.

The blog can be found via a link right pane on www.netapp.com

Hugo (NetApp employee)

-----Original Message----- From: Page, Jeremy jeremy.page@gilbarco.com To: toasters@mathworks.com toasters@mathworks.com Sent: Thu Sep 27 14:26:47 2007 Subject: NFS and VMware

Are any of the toaster folks using NFS for VMware in a production environment? We have about 150 VMs across 12 servers at the moment sitting on an IBM DS4500, but I just got my budget approved to buy a nice 3040 cluster. For ease of backup/restore/DR I really would like to use NFS instead of FCP/iSCSI but I don't know what the performance impact will be. Any comments/experiences?

This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.