thanks. how does direct access restore get figure into this issue?

Stephen Manley wrote:

...

hi,

we got legato to do a backup and now we are trying a restore. i deleted a directory from the filer and selected it for restore.

we are ontap 5.36r2, so no direct access restore.

watching the filer from an nt machine, we immediately saw the file names come back, with 0 bytes and no attributes. much later the file sizes came back and the attributes, but legato keeps reading the tape and every 5 minutes we see the "we have read so many bytes" messages.

i would think once the files are back the restore would complete, will it go to the end of the tape?

Until 5.3.7, the restore process would go to the end of the tape -- looking for ACLs, even if there were none to restore.

In 5.3.7 and beyond, if there are no ACLs to be restored, then it realizes that it is done and stops. If there are ACLs to restore on the files you selected, it will still read the whole backup.

In 6.1 and beyond, the ACLs are also at the beginning of the tape, so you won't need to go to the end of the tape at all (unless that's where your file happens to be).

As you noted - the way restore works, we create the empty files first, then fill them in with data/perms as we run through the tape.

Stephen Manley DAM and NDMP former member of Destiny's Child

hi,

i have noticed, during testing, that ndmp restores don't tell you anything[at least for the sw i have looked at]. basically, if i didn't know any better, i would think, from looking at job reports, that the whole volume or qtree was restored. the dump messages don't say anything other than restore, we have read ..., and restoring NT ACLs.

vendors say this is a limitation of ndmp. true? and that reports will be more intelligent when i move up to an ontap ver with DAR. comments?

thanks

Stephen Manley wrote:

...

thanks. how does direct access restore get figure into this issue?

There are two possibilities with DAR.

1. Via your GUI, you choose not to restore ACLs. (maybe it's easier just to set an ACL or two by hand)

In this case, we would seek to the points on the tape where your files are. At each spot it will pull off the file's data and write it to the file.

2. You choose to restore ACLs (or the GUI doesn't give you a choice)

In this case, we would seek to the points on the tape where your files are. At each spot, etc, etc, etc.

Then, when all the files are yanked off, we'd seek (not read) to the end of the backup. Then, we'd restore the ACLs.

This is the same in every DAR release.

Stephen Manley DAM and NDMP "Timmy!"

...

Stephen Manley wrote:

...

...

hi,

we got legato to do a backup and now we are trying a restore. i deleted a directory from the filer and selected it for restore.

we are ontap 5.36r2, so no direct access restore.

watching the filer from an nt machine, we immediately saw the file names come back, with 0 bytes and no attributes. much later the file sizes came back and the attributes, but legato keeps reading the tape and every 5 minutes we see the "we have read so many bytes" messages.

i would think once the files are back the restore would complete, will it go to the end of the tape?

Until 5.3.7, the restore process would go to the end of the tape -- looking for ACLs, even if there were none to restore.

In 5.3.7 and beyond, if there are no ACLs to be restored, then it realizes that it is done and stops. If there are ACLs to restore on the files you selected, it will still read the whole backup.

In 6.1 and beyond, the ACLs are also at the beginning of the tape, so you won't need to go to the end of the tape at all (unless that's where your file happens to be).

As you noted - the way restore works, we create the empty files first, then fill them in with data/perms as we run through the tape.

Stephen Manley DAM and NDMP former member of Destiny's Child

--

regards

+++++++++++++++++++++++++++++++++++++++++++++++++

• Neil Lehrer
• United States International Broadcasting Bureau
• System Development Division
• voice 202 619-2524
• fax 202 619-3576
• nlehrer@ibb.gov
• " is this crisis an opportunity or just
• another grab the fire extinguisher moment?"

+++++++++++++++++++++++++++++++++++++++++++++++++

if it takes a lot of resources then i don't know what i would ask for. but, if the report doesn't tell me what it did all i can do is eyeball the result. that might not be very accurate depending on what was being put back.

Stephen Manley wrote:

...

hi,

i have noticed, during testing, that ndmp restores don't tell you anything[at least for the sw i have looked at]. basically, if i didn't know any better, i would think, from looking at job reports, that the whole volume or qtree was restored. the dump messages don't say anything other than restore, we have read ..., and restoring NT ACLs.

vendors say this is a limitation of ndmp. true? and that reports will be more intelligent when i move up to an ontap ver with DAR. comments?

Just curious -- what sort of messages are you looking for?

The "we have read" messages are intended to let you know that restore is still making progress, and even lets you estimate whether that progress has suddenly become faster or slower.

There are options to restore by which you can make it tell you which files it is restoring. However, selecting the verbose option can slash your performance in half, or more.

Restore is integrated into Ontap, so it avoids building and parsing full pathnames, which is more expensive than operating on inodes and directories. So, while there are options to make restore print out the pathnames, it becomes extra work (heavy work) that restore does in addition to its primary task - getting your data back.

However, if you REALLY want to enable verbosity, and accept the performance penalty, there is an NDMP environment variable -- VERBOSE. I believe your NDMP client vendor might be able help you enable that.

Of course, restore will holler if it encounters errors (e.g. out of inodes, disk space, ACLs can't be restored, etc)

Is there specific info that you'd like to see that is less expensive "we're restoring this file" for each and every file, but more informative than "we've read 100GB"?

Remember this, my friend: Restore is like a engineer.

If you divert an engineer from his primary job too much, he'll end up as a marketing guy. If you divert restore too much, it'll end up talking your ears off all day with random useless information, as well...

Stephen Manley DAM and NDMP's answer to Ben Franklin

...

Stephen Manley wrote:

...

...

thanks. how does direct access restore get figure into this issue?

There are two possibilities with DAR.

1. Via your GUI, you choose not to restore ACLs. (maybe it's easier just to set an ACL or two by hand)

In this case, we would seek to the points on the tape where your files are. At each spot it will pull off the file's data and write it to the file.

2. You choose to restore ACLs (or the GUI doesn't give you a choice)

In this case, we would seek to the points on the tape where your files are. At each spot, etc, etc, etc.

Then, when all the files are yanked off, we'd seek (not read) to the end of the backup. Then, we'd restore the ACLs.

This is the same in every DAR release.

Stephen Manley DAM and NDMP "Timmy!"

...

Stephen Manley wrote:

...

...

hi,

we got legato to do a backup and now we are trying a restore. i deleted a directory from the filer and selected it for restore.

we are ontap 5.36r2, so no direct access restore.

watching the filer from an nt machine, we immediately saw the file names come back, with 0 bytes and no attributes. much later the file sizes came back and the attributes, but legato keeps reading the tape and every 5 minutes we see the "we have read so many bytes" messages.

i would think once the files are back the restore would complete, will it go to the end of the tape?

Until 5.3.7, the restore process would go to the end of the tape -- looking for ACLs, even if there were none to restore.

In 5.3.7 and beyond, if there are no ACLs to be restored, then it realizes that it is done and stops. If there are ACLs to restore on the files you selected, it will still read the whole backup.

In 6.1 and beyond, the ACLs are also at the beginning of the tape, so you won't need to go to the end of the tape at all (unless that's where your file happens to be).

As you noted - the way restore works, we create the empty files first, then fill them in with data/perms as we run through the tape.

Stephen Manley DAM and NDMP former member of Destiny's Child

--

regards

+++++++++++++++++++++++++++++++++++++++++++++++++

• Neil Lehrer
• United States International Broadcasting Bureau
• System Development Division
• voice 202 619-2524
• fax 202 619-3576
• nlehrer@ibb.gov
• " is this crisis an opportunity or just
• another grab the fire extinguisher moment?"

+++++++++++++++++++++++++++++++++++++++++++++++++

--

regards

+++++++++++++++++++++++++++++++++++++++++++++++++

• Neil Lehrer
• United States International Broadcasting Bureau
• System Development Division
• voice 202 619-2524
• fax 202 619-3576
• nlehrer@ibb.gov
• " is this crisis an opportunity or just
• another grab the fire extinguisher moment?"

+++++++++++++++++++++++++++++++++++++++++++++++++

that's a lot better. maybe add "you asked me to restore xxxxx" and give the restore request, then some stats on what actually happened.

i'd like to hear what other people have to say about this also.

Stephen Manley wrote:

...

if it takes a lot of resources then i don't know what i would ask for. but if the report doesn't tell me what it did all i can do is eyeball the result. That might not be very accurate depending on what was being put back.

Hmmm...

We could change restore to publish a cumulative result at the end:

Restored 1438093 files. Restored 32192234 MB of data. 4 files - A, B, C, D could not be restored. 63KB of data could not be restored to file "X" -- tape read error. 16 ACLs could not be set on ...

Would this be helpful? Anybody else have opinions on what they might like to see?

Printing out cumulative results shouldn't hurt performance, and thus is more attractive to a performance weenie like me.

And that way you wouldn't have to troll through the logs of a restore -- just look at the end.

I can file an RFE for this, if you'd like.

Stephen Manley DAM and NDMP Genie in a Bottle