Jeremy,
If you consider that the storage industry as having 5 main providers, IBM, EMC, HP, HDS, & NetApp, you will note that only NetApp provides NFS on their primary storage arrays. I did not list IBM N-Series (NetApp OEM) arrays as they are not IBM's primary array (which is the Shark). In addition, NFS has only been an option in the VMware space for ~18 months. So to begin NFS has a much smaller footprint due to vendor offerings and historical choices with implementation. With that said it has a lot of momentum.
On the surface the NFS (vs FCP) performance concerns seem logical. What we finding is that with virtualization, and more specifically the components of, these aren't real issues.
Does FCP outperform NFS? Yes – when the measurement is how fast can 1 server access one area of storage Yes – when the measurement is how fast can 1 virtual machine access on Datastore No – when the measurement is how much IO can an individual Datastore push to multiple VMs on multiple ESX hosts. This is why compute grids are NFS and not FCP
Does NFS consume more CPU than FCP? Yes – but our IO benchmarks show that number is ~ 20%, or in other words, CPU are powerful and abundant today, so its not an issue.
Is NFS less expensive than FCP? Yes – no switches, optical cables, HBAs
Does NFS provide greater storage utilization? Yes – VMDKs are thin provisioned Yes, - No dead space residing in VMFS file systems. NFS removes one layer in the storage stack, allowing VMware to treat disks as files, (which is what VMFS does).
In additon NFS significantly simplifies many operational tasks. Consider the requirements to grow a Datastore. On FCP you create a LUN, map it, scan for it on ESX, add a VMFS extent, and then scan for the new LUN and extent on all remaining ESX servers. With NFS on NetApp its one command and you're done, oh and I forgot, you can also shrink NFS just as easily. Try that with any SAN file system.
Our founder, Dave Hitz, went to Berkley with Mendel Rosenblum, the founder of VMware. As a grad student Mendel worked on a file system project named LFS. Dave, worked on ideas for WAFL while at Berkley and learned from the LFS project. The two met last week, and Dave asked Mendel his thoughts on NFS, to which Mendel replied, 'NetApp will always take storage virtualization to places where we will never go' and ' VMware needed VMFS in order to treat storage as files thus enabling features like VMotion.' Some of this was posted on Dave's blog: http://blogs.netapp.com/dave/
So, I would ask if NFS performs on par with FCP in a scaled out model, than doesn't simplicity and higher utilization rule in favor of NFS? My intention is not to start a protocol war of FCP vs NFS rather I am merely asking the question...
Vaughn Stewart Virtualization Evangelist NetApp
P.S. try a benchmark like I described...
On 9/27/07, Page, Jeremy jeremy.page@gilbarco.com wrote:
Thanks, I read Dave's blog and the link to the SE that was at the bottom. Some good information there, but no hard numbers. I'd really like to talk to someone that has both in production or maybe done testing on the same box with iSCSI and NFS, mainly because I'm too lazy to do it properly myself.
*From:* Doucet, Hugo [mailto:hdoucet@netapp.com] *Sent:* Thursday, September 27, 2007 10:37 AM *To:* Page, Jeremy; toasters@mathworks.com *Subject:* Re: NFS and VMware
Jeremy
I suggest you consult this weeks posts on Dave Hitz blog (NetApp founder) and people's comments.
The blog can be found via a link right pane on www.netapp.com
Hugo (NetApp employee)
-----Original Message----- From: Page, Jeremy jeremy.page@gilbarco.com To: toasters@mathworks.com toasters@mathworks.com Sent: Thu Sep 27 14:26:47 2007 Subject: NFS and VMware
Are any of the toaster folks using NFS for VMware in a production environment? We have about 150 VMs across 12 servers at the moment sitting on an IBM DS4500, but I just got my budget approved to buy a nice 3040 cluster. For ease of backup/restore/DR I really would like to use NFS instead of FCP/iSCSI but I don't know what the performance impact will be. Any comments/experiences?
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.