I was always under the impression you wanted to put as many disks into an aggr, then setup the volumes inside the aggr.
Here is what we are doing, and I am just trying to see what others are doing, best practices, etc.
We have an AGGR that we use for VMware sitting on a 6030 running 7.2.4 FCP. The aggr is ~ 9TB or so, usable and we have 1 volume that is ~ 5TB or so. We do not currenlty use SMVI (waiting for the budget to open) :) but we will use SMVI and hopefully snapmirror over to another filer. All VM's are windows (for the most part) The primary storage is still on 7.2.4, so no dedupe, or thin provisioning until we upgrade to 7.3.2 sometime this year. This is coming btw to a store near you...
Pros/cons to creating smaller/bigger volumes: potentially easier to snapmirror less dedupe % if we create smaller volumes (less vm's, less dedupe)
Is there anything wrong with creating 1 volume to 1 aggr and grow the volume to 80%? Am I missing something here?
We are thinking of reducing our snap reserve as well.
Yes, I have read the vmware best practices doc. VERY GOOD BTW.
Thanks in advance
Steve,
That's really an ESX question, not storage.
Using FC, the volume design with be based on how locks are implemented in VMFS. VMWare do offer best practice numbers which, along with your VMDK sizes, will determine the optimum size of your volumes.
When we used FC, I think we aimed for 12VM's per store, and 150GB vols.
You'll obviously get potentially better dedup savings on larger volumes. Have you considered using NFS rather than FC (which uses it's own locking mechanism so the same constraints do not apply)?
Having said all that, I think SMVI generally works better with non-NFS stores; we're still having occasional issues due to NFS related issues.
Darren
On 05/03/2009 18:44, "steve klise" klises@caminomedical.org wrote:
I was always under the impression you wanted to put as many disks into an aggr, then setup the volumes inside the aggr.
Here is what we are doing, and I am just trying to see what others are doing, best practices, etc.
We have an AGGR that we use for VMware sitting on a 6030 running 7.2.4 FCP. The aggr is ~ 9TB or so, usable and we have 1 volume that is ~ 5TB or so. We do not currenlty use SMVI (waiting for the budget to open) :) but we will use SMVI and hopefully snapmirror over to another filer. All VM's are windows (for the most part) The primary storage is still on 7.2.4, so no dedupe, or thin provisioning until we upgrade to 7.3.2 sometime this year. This is coming btw to a store near you...
Pros/cons to creating smaller/bigger volumes: potentially easier to snapmirror less dedupe % if we create smaller volumes (less vm's, less dedupe)
Is there anything wrong with creating 1 volume to 1 aggr and grow the volume to 80%? Am I missing something here?
We are thinking of reducing our snap reserve as well.
Yes, I have read the vmware best practices doc. VERY GOOD BTW.
Thanks in advance
I run Ontap 7.2.4 with thin provisioning, dedupe, on FC and SMVI works like a charm. We have VM per lun so that we backup and restore at the lun level.
Thanks, Tres
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Darren Sykes Sent: Thursday, March 05, 2009 2:18 PM To: steve klise; toasters@mathworks.com Subject: Re: vmware ESX and volumes/aggrs
Steve,
That's really an ESX question, not storage.
Using FC, the volume design with be based on how locks are implemented in VMFS. VMWare do offer best practice numbers which, along with your VMDK sizes, will determine the optimum size of your volumes.
When we used FC, I think we aimed for 12VM's per store, and 150GB vols.
You'll obviously get potentially better dedup savings on larger volumes. Have you considered using NFS rather than FC (which uses it's own locking mechanism so the same constraints do not apply)?
Having said all that, I think SMVI generally works better with non-NFS stores; we're still having occasional issues due to NFS related issues.
Darren
On 05/03/2009 18:44, "steve klise" klises@caminomedical.org wrote:
I was always under the impression you wanted to put as many disks into
an
aggr, then setup the volumes inside the aggr.
Here is what we are doing, and I am just trying to see what others are doing, best practices, etc.
We have an AGGR that we use for VMware sitting on a 6030 running 7.2.4
FCP.
The aggr is ~ 9TB or so, usable and we have 1 volume that is ~ 5TB or
so.
We do not currenlty use SMVI (waiting for the budget to open) :) but
we
will use SMVI and hopefully snapmirror over to another filer. All
VM's are
windows (for the most part) The primary storage is still on 7.2.4, so no dedupe, or thin
provisioning
until we upgrade to 7.3.2 sometime this year. This is coming btw to a
store
near you...
Pros/cons to creating smaller/bigger volumes: potentially easier to snapmirror less dedupe % if we create smaller volumes (less vm's, less dedupe)
Is there anything wrong with creating 1 volume to 1 aggr and grow the
volume
to 80%? Am I missing something here?
We are thinking of reducing our snap reserve as well.
Yes, I have read the vmware best practices doc. VERY GOOD BTW.
Thanks in advance
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Darren Sykes wrote:
Having said all that, I think SMVI generally works better with non-NFS stores; we're still having occasional issues due to NFS related issues.
Darren --
Can you elaborate on those issues for those of us eyeing both NFS for datastores and SMVI for snap management/operations?
Thanks.
- -- Nick Silkey
Yes, the issues we're seeing are that some machines lock up when undergoing SMVI backups (and not the normal bug that affects removing snapshots from VMs on NFS storage).
The latest theory is that the fixes in the latest version of VMWare tools sorts it out. VMware don't have a firm idea of what's causing it in our environment, but we've seen numerous bugs based on the locking mechanism (which is different in NFS to iSCSI or FC).
Darren.
-----Original Message----- From: Nick Silkey [mailto:silkey@ece.utexas.edu] Sent: 06 March 2009 15:31 To: Darren Sykes Cc: steve klise; toasters@mathworks.com Subject: Re: vmware ESX and volumes/aggrs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Darren Sykes wrote:
Having said all that, I think SMVI generally works better with non-NFS stores; we're still having occasional issues due to NFS related
issues.
Darren --
Can you elaborate on those issues for those of us eyeing both NFS for datastores and SMVI for snap management/operations?
Thanks.
- -- Nick Silkey
To report this email as spam click https://www.mailcontrol.com/sr/kh8Fd1nb5kHTndxI!oX7UqcUCtlg0TD8A1BaxxyeY dpMmHvx8qkAiLQ5RK2G1tICMbw0NZTqqAAwyNC8RpgWkw== .
Darren,
Do you have any details on the bug that affects removing snapshots from VM's on NFS ?
And do you have any details about which version of VM tools seems to solve the problem, I thought we were upto date with versions and patches, but still seem to seeing similar problems with smvi and NFS datastores.
Thanks
Matt
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Darren Sykes Sent: 06 March 2009 17:51 To: Nick Silkey Cc: steve klise; toasters@mathworks.com Subject: RE: vmware ESX and volumes/aggrs
Yes, the issues we're seeing are that some machines lock up when undergoing SMVI backups (and not the normal bug that affects removing snapshots from VMs on NFS storage).
The latest theory is that the fixes in the latest version of VMWare tools sorts it out. VMware don't have a firm idea of what's causing it in our environment, but we've seen numerous bugs based on the locking mechanism (which is different in NFS to iSCSI or FC).
Darren.
-----Original Message----- From: Nick Silkey [mailto:silkey@ece.utexas.edu] Sent: 06 March 2009 15:31 To: Darren Sykes Cc: steve klise; toasters@mathworks.com Subject: Re: vmware ESX and volumes/aggrs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Darren Sykes wrote:
Having said all that, I think SMVI generally works better with non-NFS stores; we're still having occasional issues due to NFS related
issues.
Darren --
Can you elaborate on those issues for those of us eyeing both NFS for datastores and SMVI for snap management/operations?
Thanks.
- -- Nick Silkey
To report this email as spam click https://www.mailcontrol.com/sr/kh8Fd1nb5kHTndxI!oX7UqcUCtlg0TD8A1BaxxyeY dpMmHvx8qkAiLQ5RK2G1tICMbw0NZTqqAAwyNC8RpgWkw== .
_____________________________________________________________ This e-mail (including all attachments) is confidential and may be privileged. It is for the exclusive use of the addressee only. If you are not the addressee, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please erase all copies of the message and its attachments and notify us immediately at help@generalatlantic.com mailto:help@generalatlantic.com. Thank You.
Hi Matt
This is ESX350-200808401-BG. The issue has to do with how locks are handled during snapshot commits, and other functionality that uses delta files. The fix has nothing to do with VMware tools. For ESX 3.5 updates 1 and 2, you need to install ESX350-200808401-BG (or a bundle that includes it). For ESX 3.5 Update 3, the patch is already there and you just need to activate it.
To activate it, add the following line to /etc/vmware/config prefvmx.consolidateDeleteNFSLocks = "TRUE" and reboot.
Details are in TR3428 http://www.netapp.com/us/library/technical-reports/tr-3428.html
Enjoy!
Peter
-----Original Message----- From: Davies,Matt [mailto:MDAVIES@generalatlantic.com] Sent: Friday, March 06, 2009 1:10 PM To: Darren Sykes; Nick Silkey Cc: steve klise; toasters@mathworks.com Subject: RE: vmware ESX and volumes/aggrs
Darren,
Do you have any details on the bug that affects removing snapshots from VM's on NFS ?
And do you have any details about which version of VM tools seems to solve the problem, I thought we were upto date with versions and patches, but still seem to seeing similar problems with smvi and NFS datastores.
Thanks
Matt
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Darren Sykes Sent: 06 March 2009 17:51 To: Nick Silkey Cc: steve klise; toasters@mathworks.com Subject: RE: vmware ESX and volumes/aggrs
Yes, the issues we're seeing are that some machines lock up when undergoing SMVI backups (and not the normal bug that affects removing snapshots from VMs on NFS storage).
The latest theory is that the fixes in the latest version of VMWare tools sorts it out. VMware don't have a firm idea of what's causing it in our environment, but we've seen numerous bugs based on the locking mechanism (which is different in NFS to iSCSI or FC).
Darren.
-----Original Message----- From: Nick Silkey [mailto:silkey@ece.utexas.edu] Sent: 06 March 2009 15:31 To: Darren Sykes Cc: steve klise; toasters@mathworks.com Subject: Re: vmware ESX and volumes/aggrs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Darren Sykes wrote:
Having said all that, I think SMVI generally works better with non-NFS
stores; we're still having occasional issues due to NFS related
issues.
Darren --
Can you elaborate on those issues for those of us eyeing both NFS for datastores and SMVI for snap management/operations?
Thanks.
- -- Nick Silkey
To report this email as spam click https://www.mailcontrol.com/sr/kh8Fd1nb5kHTndxI!oX7UqcUCtlg0TD8A1BaxxyeY dpMmHvx8qkAiLQ5RK2G1tICMbw0NZTqqAAwyNC8RpgWkw== .
_____________________________________________________________ This e-mail (including all attachments) is confidential and may be privileged. It is for the exclusive use of the addressee only. If you are not the addressee, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please erase all copies of the message and its attachments and notify us immediately at help@generalatlantic.com mailto:help@generalatlantic.com. Thank You.
Peter,
As far as I know, there are 2 issues.
The first one is as you described.
The second is a bug in the VSS driver in VMWare tools.
However, the bugs seem to manifest in the same way i.e. snapshot removal on NFS VMs hangs for a long period of time, during which the VM is dead to the world.
Darren
-----Original Message----- From: Learmonth, Peter [mailto:Peter.Learmonth@netapp.com] Sent: 06 March 2009 21:57 To: Davies,Matt; Darren Sykes; Nick Silkey Cc: steve klise; toasters@mathworks.com Subject: RE: vmware ESX and volumes/aggrs
Hi Matt
This is ESX350-200808401-BG. The issue has to do with how locks are handled during snapshot commits, and other functionality that uses delta files. The fix has nothing to do with VMware tools. For ESX 3.5 updates 1 and 2, you need to install ESX350-200808401-BG (or a bundle that includes it). For ESX 3.5 Update 3, the patch is already there and you just need to activate it.
To activate it, add the following line to /etc/vmware/config prefvmx.consolidateDeleteNFSLocks = "TRUE" and reboot.
Details are in TR3428 http://www.netapp.com/us/library/technical-reports/tr-3428.html
Enjoy!
Peter
-----Original Message----- From: Davies,Matt [mailto:MDAVIES@generalatlantic.com] Sent: Friday, March 06, 2009 1:10 PM To: Darren Sykes; Nick Silkey Cc: steve klise; toasters@mathworks.com Subject: RE: vmware ESX and volumes/aggrs
Darren,
Do you have any details on the bug that affects removing snapshots from VM's on NFS ?
And do you have any details about which version of VM tools seems to solve the problem, I thought we were upto date with versions and patches, but still seem to seeing similar problems with smvi and NFS datastores.
Thanks
Matt
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Darren Sykes Sent: 06 March 2009 17:51 To: Nick Silkey Cc: steve klise; toasters@mathworks.com Subject: RE: vmware ESX and volumes/aggrs
Yes, the issues we're seeing are that some machines lock up when undergoing SMVI backups (and not the normal bug that affects removing snapshots from VMs on NFS storage).
The latest theory is that the fixes in the latest version of VMWare tools sorts it out. VMware don't have a firm idea of what's causing it in our environment, but we've seen numerous bugs based on the locking mechanism (which is different in NFS to iSCSI or FC).
Darren.
-----Original Message----- From: Nick Silkey [mailto:silkey@ece.utexas.edu] Sent: 06 March 2009 15:31 To: Darren Sykes Cc: steve klise; toasters@mathworks.com Subject: Re: vmware ESX and volumes/aggrs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Darren Sykes wrote:
Having said all that, I think SMVI generally works better with non-NFS
stores; we're still having occasional issues due to NFS related
issues.
Darren --
Can you elaborate on those issues for those of us eyeing both NFS for datastores and SMVI for snap management/operations?
Thanks.
- -- Nick Silkey
To report this email as spam click https://www.mailcontrol.com/sr/kh8Fd1nb5kHTndxI!oX7UqcUCtlg0TD8A1BaxxyeY dpMmHvx8qkAiLQ5RK2G1tICMbw0NZTqqAAwyNC8RpgWkw== .
_____________________________________________________________ This e-mail (including all attachments) is confidential and may be privileged. It is for the exclusive use of the addressee only. If you are not the addressee, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please erase all copies of the message and its attachments and notify us immediately at help@generalatlantic.com mailto:help@generalatlantic.com. Thank You.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
steve klise wrote:
The primary storage is still on 7.2.4, so no dedupe, or thin provisioning until we upgrade to 7.3.2 sometime this year. This is coming btw to a store near you...
If youre on a 3xxx/6xxx FAS-series and running DOT >= 7.2.4, you can take advantage of the special sauce. You will have to obtain one of those no-cost NearStore licenses (made available as a free offering on 3/10/08) to get dedupe enabled. We went through a zero-dollar purchase order route as an edu a year ago. Dont know if this process has changed or not since; reach out to your NetApp contacts to touch base on this.
- -- Nick Silkey
I thought the current minimum supported release if 7.2.5.1?
There's also the PITA NDMP backup issues on fat VMDKs which wasn't fixed until 7.2.6.1.
Darren
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Nick Silkey Sent: 06 March 2009 15:36 To: steve klise Cc: toasters@mathworks.com Subject: Re: vmware ESX and volumes/aggrs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
steve klise wrote:
The primary storage is still on 7.2.4, so no dedupe, or thin
provisioning
until we upgrade to 7.3.2 sometime this year. This is coming btw to a
store
near you...
If youre on a 3xxx/6xxx FAS-series and running DOT >= 7.2.4, you can take advantage of the special sauce. You will have to obtain one of those no-cost NearStore licenses (made available as a free offering on 3/10/08) to get dedupe enabled. We went through a zero-dollar purchase order route as an edu a year ago. Dont know if this process has changed or not since; reach out to your NetApp contacts to touch base on this.
- -- Nick Silkey
To report this email as spam click https://www.mailcontrol.com/sr/6c2S0J1yqkfTndxI!oX7Uj80y4Ou3KxpIJyK6yk3g J3Ede4Yaq0oa3LaL3tNWGSDMbw0NZTqqAB6aunJTqVygQ== .
I strongly recommend NOT using ASIS until you're at least at 7.2.6.1/7.3.1 Before 7.2.6.1 or 7.3.1 you'll run into a problem doing sequential reads of large empty files (LUNs or VMDK files for instance). It's an ugly situation, my snapvault initial transfers for 1 TB volumes where taking a week+ to run when the 5 TB volume (not deduped) went overnight.
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Darren Sykes Sent: Friday, March 06, 2009 12:28 PM To: Nick Silkey; steve klise Cc: toasters@mathworks.com Subject: RE: vmware ESX and volumes/aggrs
I thought the current minimum supported release if 7.2.5.1?
There's also the PITA NDMP backup issues on fat VMDKs which wasn't fixed until 7.2.6.1.
Darren
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Nick Silkey Sent: 06 March 2009 15:36 To: steve klise Cc: toasters@mathworks.com Subject: Re: vmware ESX and volumes/aggrs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
steve klise wrote:
The primary storage is still on 7.2.4, so no dedupe, or thin
provisioning
until we upgrade to 7.3.2 sometime this year. This is coming btw to a
store
near you...
If youre on a 3xxx/6xxx FAS-series and running DOT >= 7.2.4, you can take advantage of the special sauce. You will have to obtain one of those no-cost NearStore licenses (made available as a free offering on 3/10/08) to get dedupe enabled. We went through a zero-dollar purchase order route as an edu a year ago. Dont know if this process has changed or not since; reach out to your NetApp contacts to touch base on this.
- -- Nick Silkey
To report this email as spam click https://www.mailcontrol.com/sr/6c2S0J1yqkfTndxI!oX7Uj80y4Ou3KxpIJyK6yk3g J3Ede4Yaq0oa3LaL3tNWGSDMbw0NZTqqAB6aunJTqVygQ== .
Please be advised that this email may contain confidential information. If you are not the intended recipient, please do not read, copy or re-transmit this email. If you have received this email in error, please notify us by email by replying to the sender and by telephone (call us collect at +1 202-828-0850) and delete this message and any attachments. Thank you in advance for your cooperation and assistance.
In addition, Danaher and its subsidiaries disclaim that the content of this email constitutes an offer to enter into, or the acceptance of, any contract or agreement or any amendment thereto; provided that the foregoing disclaimer does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment to this email.