Hello, I have what could be a major problem soon. I have a snapshot that was created by an ndmpd session, that cannot be deleted. It looks like the backup must have failed, but the netapp thinks it is still running. Any ideas on how to delete the snapshot_for_backup.19?
dice-f740-1> snap list Volume vol0 working...............................................
%/used %/total date name ---------- ---------- ------------ -------- 2% ( 2%) 1% ( 1%) Nov 29 08:07 hourly.0 7% ( 5%) 4% ( 3%) Nov 29 00:08 nightly.0 9% ( 5%) 6% ( 3%) Nov 28 20:29 oraclebackup 10% ( 5%) 7% ( 3%) Nov 28 20:04 hourly.1 14% ( 6%) 9% ( 4%) Nov 28 16:04 hourly.2 16% ( 6%) 12% ( 4%) Nov 28 12:07 hourly.3 21% ( 9%) 16% ( 6%) Nov 28 08:05 hourly.4 24% (10%) 19% ( 7%) Nov 28 00:06 nightly.1 26% (10%) 21% ( 7%) Nov 27 20:06 hourly.5 31% (11%) 26% ( 8%) Nov 23 21:04 snapshot_for_backup.19 (busy) dice-f740-1> snap delete snapshot_for_backup.19 Snapshot in use by dump or a CIFS share. dice-f740-1>
Thanks, Brian
BrianH@dice.com writes:
Hello, I have what could be a major problem soon. I have a snapshot that was created by an ndmpd session, that cannot be deleted. It looks like the backup must have failed, but the netapp thinks it is still running. Any ideas on how to delete the snapshot_for_backup.19?
dice-f740-1> snap list
[...]
31% (11%) 26% ( 8%) Nov 23 21:04 snapshot_for_backup.19 (busy) dice-f740-1> snap delete snapshot_for_backup.19 Snapshot in use by dump or a CIFS share.S
See the toasters thread "Deleting snapshot" in early September. Supposing that "ndmpd status" shows the ndmpd session gone, then this would seem to be the same problem. Stephen Manley told us then that the problem was probably 20847, which is fixed in OnTap 5.3.6. What release are you using?
If that is your problem, you will have to reboot before you can delete the snapshot. :-(
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
BrianH@dice.com writes:
Hello, I have what could be a major problem soon. I have a snapshot that was created by an ndmpd session, that cannot be deleted. It looks like the backup must have failed, but the netapp thinks it is still running. Any ideas on how to delete the snapshot_for_backup.19?
dice-f740-1> snap list
[...]
31% (11%) 26% ( 8%) Nov 23 21:04 snapshot_for_backup.19 (busy) dice-f740-1> snap delete snapshot_for_backup.19 Snapshot in use by dump or a CIFS share.S
See the toasters thread "Deleting snapshot" in early September. Supposing that "ndmpd status" shows the ndmpd session gone, then this would seem to be the same problem. Stephen Manley told us then that the problem was probably 20847, which is fixed in OnTap 5.3.6. What release are you using?
If that is your problem, you will have to reboot before you can delete the snapshot. :-(
Actually, 31340 is the preferred reference to this bug now, not 20847.
There's lots of interesting reasons for this.
Hmmm. No, actually, there aren't.
When you call CS, just refer to burt 31340. It'll save them the time/trouble of mapping from 20847 to 31340.
Stephen Manley DAM and NDMP answer to Marty McFly
Hello, I have what could be a major problem soon. I have a snapshot that was created by an ndmpd session, that cannot be deleted. It looks like the backup must have failed, but the netapp thinks it is still running. Any ideas on how to delete the snapshot_for_backup.19?
dice-f740-1> snap list Volume vol0 working...............................................
%/used %/total date name
2% ( 2%) 1% ( 1%) Nov 29 08:07 hourly.0 7% ( 5%) 4% ( 3%) Nov 29 00:08 nightly.0 9% ( 5%) 6% ( 3%) Nov 28 20:29 oraclebackup 10% ( 5%) 7% ( 3%) Nov 28 20:04 hourly.1 14% ( 6%) 9% ( 4%) Nov 28 16:04 hourly.2 16% ( 6%) 12% ( 4%) Nov 28 12:07 hourly.3 21% ( 9%) 16% ( 6%) Nov 28 08:05 hourly.4 24% (10%) 19% ( 7%) Nov 28 00:06 nightly.1 26% (10%) 21% ( 7%) Nov 27 20:06 hourly.5 31% (11%) 26% ( 8%) Nov 23 21:04 snapshot_for_backup.19 (busy) dice-f740-1> snap delete snapshot_for_backup.19 Snapshot in use by dump or a CIFS share. dice-f740-1>
Hi everybody! Dr. Nick Riviera here!
I think Brian is actually hitting burt 31340:
%%% TITLE: ndmpd deadlock during aborts can cause a locked snapshot %%% DESCRIPTION: Aborting an ndmpd session could cause a locked snapshot to occur. The ndmpd session tries to abort the dump by repeatedly sending it a signal until the signal gets through. However, the dump is waiting for data being sent to the ndmpd session to drain first before processing the abort signal. This results in a deadlock situation where the dump thread does not terminate to release the snapshot. %%% WORKAROUND: Reboot the filer.
This is a filer bug. It is caused by using NDMP, but it is not the NDMP client's fault.
I believe if you contact customer support, you should be able to get a patch for this particular problem.
After getting the fix for 31340, no customer has had a problem with a snapshot being locked after the dump/NDMP has been terminated.
As for Bruce's request for a command to terminate dumps - we agree with you, and we're working on it.
So, I hope this helps.
Stephen Manley DAM and NDMP Mascot
-
BrianH＠dice.com -
Chris Thompson -
Stephen Manley