For the folks using NFS to hold their VMDK files, have you run into a limit to how many VM's a single share can support? Either a hard limit or just a point where performance degrades? I'm trying to figure out how many volumes I'll need to create to hold ~300 VMs on my 3070 cluster.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
http://media.netapp.com/documents/tr-3428.pdf http://media.netapp.com/documents/tr-3428.pdf
See the above TR for best practices with VMWare ESX. Lun alignment is crucial to performance with an environment that has many VMDK's.
________________________________
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Wednesday, May 28, 2008 7:10 AM To: toasters@mathworks.com Subject: Is there a limit to the number of VM's per NFS export?
For the folks using NFS to hold their VMDK files, have you run into a limit to how many VM's a single share can support? Either a hard limit or just a point where performance degrades? I'm trying to figure out how many volumes I'll need to create to hold ~300 VMs on my 3070 cluster.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
I'm not using LUNs, I'm using NFS, which unfortunately is not very well covered in that document.
Jeremy M. Page____________________
Systems Architect
* email:Jeremy.Page@gilbarco.com - * phone: 336.547.5399 - 6 fax: 336.547.5163 - * cell: 336.601.7274
________________________________
From: King, Robert [mailto:Robert.King@netapp.com] Sent: Wednesday, May 28, 2008 8:29 AM To: Page, Jeremy; toasters@mathworks.com Subject: RE: Is there a limit to the number of VM's per NFS export?
http://media.netapp.com/documents/tr-3428.pdf http://media.netapp.com/documents/tr-3428.pdf
See the above TR for best practices with VMWare ESX. Lun alignment is crucial to performance with an environment that has many VMDK's.
________________________________
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Wednesday, May 28, 2008 7:10 AM To: toasters@mathworks.com Subject: Is there a limit to the number of VM's per NFS export?
For the folks using NFS to hold their VMDK files, have you run into a limit to how many VM's a single share can support? Either a hard limit or just a point where performance degrades? I'm trying to figure out how many volumes I'll need to create to hold ~300 VMs on my 3070 cluster.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
There is a section that covers offset's for the vmdk's. This is particularly important when implementing vmdk's in order to avoid partial writes.
________________________________
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Wednesday, May 28, 2008 8:41 AM To: King, Robert; toasters@mathworks.com Subject: RE: Is there a limit to the number of VM's per NFS export?
I'm not using LUNs, I'm using NFS, which unfortunately is not very well covered in that document.
Jeremy M. Page____________________
Systems Architect
* email:Jeremy.Page@gilbarco.com - * phone: 336.547.5399 - 6 fax: 336.547.5163 - * cell: 336.601.7274
________________________________
From: King, Robert [mailto:Robert.King@netapp.com] Sent: Wednesday, May 28, 2008 8:29 AM To: Page, Jeremy; toasters@mathworks.com Subject: RE: Is there a limit to the number of VM's per NFS export?
http://media.netapp.com/documents/tr-3428.pdf http://media.netapp.com/documents/tr-3428.pdf
See the above TR for best practices with VMWare ESX. Lun alignment is crucial to performance with an environment that has many VMDK's.
________________________________
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Wednesday, May 28, 2008 7:10 AM To: toasters@mathworks.com Subject: Is there a limit to the number of VM's per NFS export?
For the folks using NFS to hold their VMDK files, have you run into a limit to how many VM's a single share can support? Either a hard limit or just a point where performance degrades? I'm trying to figure out how many volumes I'll need to create to hold ~300 VMs on my 3070 cluster.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
I think you will want to address this in terms of the 'ops'. Start with ensuring that you are employing all the NFS best practices and make sure your NFS host (in this case the filer) scales to the load you intend.
Stetson M. Webster Onsite Professional Services Engineer PS - North Amer. - East
NetApp 919.250.0052 Mobile Stetson.Webster@netapp.com www.netapp.com http://www.netapp.com/
________________________________
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Wednesday, May 28, 2008 8:41 AM To: King, Robert; toasters@mathworks.com Subject: RE: Is there a limit to the number of VM's per NFS export?
I'm not using LUNs, I'm using NFS, which unfortunately is not very well covered in that document.
Jeremy M. Page____________________
Systems Architect
* email:Jeremy.Page@gilbarco.com - * phone: 336.547.5399 - 6 fax: 336.547.5163 - * cell: 336.601.7274
________________________________
From: King, Robert [mailto:Robert.King@netapp.com] Sent: Wednesday, May 28, 2008 8:29 AM To: Page, Jeremy; toasters@mathworks.com Subject: RE: Is there a limit to the number of VM's per NFS export?
http://media.netapp.com/documents/tr-3428.pdf http://media.netapp.com/documents/tr-3428.pdf
See the above TR for best practices with VMWare ESX. Lun alignment is crucial to performance with an environment that has many VMDK's.
________________________________
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Wednesday, May 28, 2008 7:10 AM To: toasters@mathworks.com Subject: Is there a limit to the number of VM's per NFS export?
For the folks using NFS to hold their VMDK files, have you run into a limit to how many VM's a single share can support? Either a hard limit or just a point where performance degrades? I'm trying to figure out how many volumes I'll need to create to hold ~300 VMs on my 3070 cluster.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
I'm not using LUNs, I'm using NFS, which unfortunately is not very well covered in that document.
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Wednesday, May 28, 2008 7:10 AM To: toasters@mathworks.com Subject: Is there a limit to the number of VM's per NFS export?
For the folks using NFS to hold their VMDK files, have you run into a limit to how many VM's a single share can support? Either a hard limit or just a point where performance degrades? I'm trying to figure out how many volumes I'll need to create to hold ~300 VMs on my 3070 cluster.
Using multiple volumes is more for snapshot and dedupe management, since these are volume wide operations.
I don't think that there is any need to split VMDKs over multiple volumes simply for NFS performance reasons.
From what I've heard/read you should have (at least) three volumes, each with a different snapshot/dedupe configuration. Each VM will have some of its VMDKs in each of these volumes, so each VM has at least 3 VMDKs.
One volume should be for VMDKs that hold swap space and temporary files. This volume should not have snapshots enabled. Since swap space is highly volatile, and never needs to be restored, mixing it with less volatile data in the same volume needlessly consumes snapshot space. It probably doesn't pay to deduplicate this volume either.
One volume should be for VMDKs to hold the guest OS install. This volume should have snapshots enabled, but perhaps not very often. This volume should definitely use deduplication since you will probably be installing the same guest OS on very many VMs.
One volume should be for VMDKs that hold VM local software and data. This volume should have snapshots enabled to occur more frequently, since this is the data that will most likely need to be restored from snapshots.
Steve Losen scl@virginia.edu phone: 434-924-0640
University of Virginia ITC Unix Support
-
King, Robert -
Page, Jeremy -
Stephen C. Losen -
Webster, Stetson