We just moved these VMs to this datastore, They are freshly rebooted after having the Vmtools updated...
-----Original Message----- From: Matt Hallmark [mailto:matt@cosmictoaster.com] Sent: Wednesday, May 20, 2009 4:51 PM To: Sto Rage(c) Cc: Ken Williams; toasters@mathworks.com Subject: Re: SMVI "Operation timed out" during queice / vmware-level snapshot
Is this a new VM, or with tools newly installed? Try rebooting the VM, that sometimes fixes these symptoms for me.
Matt Hallmark
On May 20, 2009, at 2:17 PM, Sto Rage(c) wrote:
We have seen something similar recently. Is "snqfpapp01" a Windows VM? Check the version of VMware tools installed on that VM and reinstall/upgrade if necessary. That fixed the issue for us. Hope that helps
-Sto Rage
On Wed, May 20, 2009 at 11:35 AM, Ken Williams kwillia@smud.org wrote:
Anyone seeing these types of issues?
I have a FAS3070 cluster serving VMFS datastore over fiber channel to
a 6 server cluster of ESX 3.5 latest SP. I seem to get a couple VMs (randomly) that are unable to perform the VMWare snapshot and cause the snapshot
to fail. The problem seems to be hit or miss.
Here's an excerpt from the log: 2009-05-19 19:18:47,556 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:18:57,681 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:18:57,697 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:19:07,822 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:19:07,837 WARN - VMware Task "CreateSnapshot_Task" for entity "snqfpapp01" failed with the following error: Operation timed out. 2009-05-19 19:19:07,837 ERROR - VM "snqfpapp01" will not be backed up
since vmware snapshot create operation failed.
I have a case open with Netapp but I'd like to see if the community has seen this. So has anyone else seen this problem? Any suggestions? __________________________________________________________ Ken Williams Storage Administrator, Business Technology Operations Sacramento Municipal Utility District E-Mail: kwillia@smud.org Phone: (916) 732-6744 Cell: (916) 240-4213
Not much help I know, but we see this issue on some VM's occasionally.
Are the failing VM's different from the rest? I seem to remember linking our failures to VM's with iSCSI disks in the guests.
The other thing to check is that you're running the VSS driver as part of VMWare tools (if you continually upgrade VMWare tools from older versions as we have then you may inadvertently miss the install of VSS).
Darren
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ken Williams Sent: 21 May 2009 17:18 To: Matt Hallmark; Sto RageC Cc: toasters@mathworks.com Subject: RE: SMVI "Operation timed out" during queice / vmware-level snapshot
We just moved these VMs to this datastore, They are freshly rebooted after having the Vmtools updated...
-----Original Message----- From: Matt Hallmark [mailto:matt@cosmictoaster.com] Sent: Wednesday, May 20, 2009 4:51 PM To: Sto Rage(c) Cc: Ken Williams; toasters@mathworks.com Subject: Re: SMVI "Operation timed out" during queice / vmware-level snapshot
Is this a new VM, or with tools newly installed? Try rebooting the VM, that sometimes fixes these symptoms for me.
Matt Hallmark
On May 20, 2009, at 2:17 PM, Sto Rage(c) wrote:
We have seen something similar recently. Is "snqfpapp01" a Windows VM? Check the version of VMware tools installed on that VM and reinstall/upgrade if necessary. That fixed the issue for us. Hope that helps
-Sto Rage
On Wed, May 20, 2009 at 11:35 AM, Ken Williams kwillia@smud.org wrote:
Anyone seeing these types of issues?
I have a FAS3070 cluster serving VMFS datastore over fiber channel to
a 6 server cluster of ESX 3.5 latest SP. I seem to get a couple VMs (randomly) that are unable to perform the VMWare snapshot and cause the snapshot
to fail. The problem seems to be hit or miss.
Here's an excerpt from the log: 2009-05-19 19:18:47,556 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:18:57,681 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:18:57,697 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:19:07,822 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:19:07,837 WARN - VMware Task "CreateSnapshot_Task" for entity "snqfpapp01" failed with the following error: Operation timed out. 2009-05-19 19:19:07,837 ERROR - VM "snqfpapp01" will not be backed up
since vmware snapshot create operation failed.
I have a case open with Netapp but I'd like to see if the community has seen this. So has anyone else seen this problem? Any suggestions? __________________________________________________________ Ken Williams Storage Administrator, Business Technology Operations Sacramento Municipal Utility District E-Mail: kwillia@smud.org Phone: (916) 732-6744 Cell: (916) 240-4213
To report this email as spam click https://www.mailcontrol.com/sr/qbWPulKgB9bTndxI!oX7Uu4ItyQZZf3fgvDIMwks4 !XPFJ9TGtWz1TxjQfCV8S+crFqheWj7AFz5CCz92bNxfg== .
What verions of ESX are you seeing this on? The recommendation from Vmware is to install ESX 3.5 Update 4 in order to see reduced instances of "Operation timed out".
-----Original Message----- From: Darren Sykes [mailto:Darren.Sykes@csr.com] Sent: Friday, May 22, 2009 12:40 AM To: Ken Williams; Matt Hallmark; Sto RageC Cc: toasters@mathworks.com Subject: RE: SMVI "Operation timed out" during queice / vmware-level snapshot
Not much help I know, but we see this issue on some VM's occasionally.
Are the failing VM's different from the rest? I seem to remember linking our failures to VM's with iSCSI disks in the guests.
The other thing to check is that you're running the VSS driver as part of VMWare tools (if you continually upgrade VMWare tools from older versions as we have then you may inadvertently miss the install of VSS).
Darren
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ken Williams Sent: 21 May 2009 17:18 To: Matt Hallmark; Sto RageC Cc: toasters@mathworks.com Subject: RE: SMVI "Operation timed out" during queice / vmware-level snapshot
We just moved these VMs to this datastore, They are freshly rebooted after having the Vmtools updated...
-----Original Message----- From: Matt Hallmark [mailto:matt@cosmictoaster.com] Sent: Wednesday, May 20, 2009 4:51 PM To: Sto Rage(c) Cc: Ken Williams; toasters@mathworks.com Subject: Re: SMVI "Operation timed out" during queice / vmware-level snapshot
Is this a new VM, or with tools newly installed? Try rebooting the VM, that sometimes fixes these symptoms for me.
Matt Hallmark
On May 20, 2009, at 2:17 PM, Sto Rage(c) wrote:
We have seen something similar recently. Is "snqfpapp01" a Windows VM? Check the version of VMware tools installed on that VM and reinstall/upgrade if necessary. That fixed the issue for us. Hope that helps
-Sto Rage
On Wed, May 20, 2009 at 11:35 AM, Ken Williams kwillia@smud.org wrote:
Anyone seeing these types of issues?
I have a FAS3070 cluster serving VMFS datastore over fiber channel to
a 6 server cluster of ESX 3.5 latest SP. I seem to get a couple VMs (randomly) that are unable to perform the VMWare snapshot and cause the snapshot
to fail. The problem seems to be hit or miss.
Here's an excerpt from the log: 2009-05-19 19:18:47,556 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:18:57,681 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:18:57,697 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:19:07,822 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:19:07,837 WARN - VMware Task "CreateSnapshot_Task" for entity "snqfpapp01" failed with the following error: Operation timed out. 2009-05-19 19:19:07,837 ERROR - VM "snqfpapp01" will not be backed up
since vmware snapshot create operation failed.
I have a case open with Netapp but I'd like to see if the community has seen this. So has anyone else seen this problem? Any suggestions? __________________________________________________________ Ken Williams Storage Administrator, Business Technology Operations Sacramento Municipal Utility District E-Mail: kwillia@smud.org Phone: (916) 732-6744 Cell: (916) 240-4213
To report this email as spam click https://www.mailcontrol.com/sr/qbWPulKgB9bTndxI!oX7Uu4ItyQZZf3fgvDIMwks4 !XPFJ9TGtWz1TxjQfCV8S+crFqheWj7AFz5CCz92bNxfg== .
Kirpekar,
Thanks! We're at ESX 3.5 Update 4 right now.
Darren,
I'll have the VMWare admins check the individual VMs for this. They were disabling VSS a while back... It's a possibility they did not turn it back on.
-----Original Message----- From: Kirpekar, Anupama [mailto:Anupama.Kirpekar@netapp.com] Sent: Friday, May 22, 2009 9:11 AM To: Darren Sykes; Ken Williams; Matt Hallmark; Sto RageC Cc: toasters@mathworks.com Subject: RE: SMVI "Operation timed out" during queice / vmware-level snapshot
What verions of ESX are you seeing this on? The recommendation from Vmware is to install ESX 3.5 Update 4 in order to see reduced instances of "Operation timed out".
-----Original Message----- From: Darren Sykes [mailto:Darren.Sykes@csr.com] Sent: Friday, May 22, 2009 12:40 AM To: Ken Williams; Matt Hallmark; Sto RageC Cc: toasters@mathworks.com Subject: RE: SMVI "Operation timed out" during queice / vmware-level snapshot
Not much help I know, but we see this issue on some VM's occasionally.
Are the failing VM's different from the rest? I seem to remember linking our failures to VM's with iSCSI disks in the guests.
The other thing to check is that you're running the VSS driver as part of VMWare tools (if you continually upgrade VMWare tools from older versions as we have then you may inadvertently miss the install of VSS).
Darren
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ken Williams Sent: 21 May 2009 17:18 To: Matt Hallmark; Sto RageC Cc: toasters@mathworks.com Subject: RE: SMVI "Operation timed out" during queice / vmware-level snapshot
We just moved these VMs to this datastore, They are freshly rebooted after having the Vmtools updated...
-----Original Message----- From: Matt Hallmark [mailto:matt@cosmictoaster.com] Sent: Wednesday, May 20, 2009 4:51 PM To: Sto Rage(c) Cc: Ken Williams; toasters@mathworks.com Subject: Re: SMVI "Operation timed out" during queice / vmware-level snapshot
Is this a new VM, or with tools newly installed? Try rebooting the VM, that sometimes fixes these symptoms for me.
Matt Hallmark
On May 20, 2009, at 2:17 PM, Sto Rage(c) wrote:
We have seen something similar recently. Is "snqfpapp01" a Windows VM? Check the version of VMware tools installed on that VM and reinstall/upgrade if necessary. That fixed the issue for us. Hope that helps
-Sto Rage
On Wed, May 20, 2009 at 11:35 AM, Ken Williams kwillia@smud.org wrote:
Anyone seeing these types of issues?
I have a FAS3070 cluster serving VMFS datastore over fiber channel to
a 6 server cluster of ESX 3.5 latest SP. I seem to get a couple VMs (randomly) that are unable to perform the VMWare snapshot and cause the snapshot
to fail. The problem seems to be hit or miss.
Here's an excerpt from the log: 2009-05-19 19:18:47,556 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:18:57,681 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:18:57,697 INFO - FLOW-11012: Operation requested retry 2009-05-19 19:19:07,822 INFO - FLOW-11006: Starting operation in threadpool backup 2009-05-19 19:19:07,837 WARN - VMware Task "CreateSnapshot_Task" for entity "snqfpapp01" failed with the following error: Operation timed out. 2009-05-19 19:19:07,837 ERROR - VM "snqfpapp01" will not be backed up
since vmware snapshot create operation failed.
I have a case open with Netapp but I'd like to see if the community has seen this. So has anyone else seen this problem? Any suggestions? __________________________________________________________ Ken Williams Storage Administrator, Business Technology Operations Sacramento Municipal Utility District E-Mail: kwillia@smud.org Phone: (916) 732-6744 Cell: (916) 240-4213
To report this email as spam click https://www.mailcontrol.com/sr/qbWPulKgB9bTndxI!oX7Uu4ItyQZZf3fgvDIMwks4 !XPFJ9TGtWz1TxjQfCV8S+crFqheWj7AFz5CCz92bNxfg== .