I started testing sis on one of our 3040's last weekend. Sorry if theres more extensive information on NOW, but the things I've found so far were fairly rudimentry. Some impressions and questions:
I found out quickly that my 5T test volume was too big. Looked up the limits because I had no idea there were volume size limits. 3T for 3040 hmm. I have an existing volume that holds 2.5T, so that is pushing it, and while I would like to split it up, it won't happen overnight. I'm not desperate to use sis on it, although I am almost done copying it and so far have realized 26% savings from dedupe for testing. I wonder why the max size scales with the system model, I haven't thought of a good reason for this yet since you could easily have lots of 3T volumes. I was wondering if you can get sis to shrink say 2.8T to 2.0, is the 2.0 what counts for the sis volume limit size? Even so, would I run into trouble if trying to do a full volume restore and the full data set is over 3.0 but would have shrunk? Once I write over 3T into a volume, it sounds like I cannot run sis on it. I'm not too worried about having to do a full restore of a volume from tape, but I don't want to limit my future options if the short term payoff isn't worth it. I'm pretty sure I'll use it on all or most of all of my other volumes since they are much smaller.
I noticed during my data copy, the snapshot reserve was blowing out big time. It seemed like almost any savings from sis went into snapshots for some reason. 100, 200, 400% full, and it started trimming itself back after a while. Since this was a non-production copy, I didn't care, and deleted those snapshots this morning, but I thought that was rather odd, and couldn't think what the snapshots would have to offer since if the data was unmodified at the filesystem level, the snapshot should contain the same data. While not a problem for an initial copy, I would expect the same thing to happen when sis runs during production, and although it wouldn't be as large and would eventually flush out, why am I expending space to store duplicate copies of data that I asked it to deduplicate? :) Maybe its just because wafl identifies them as "changed blocks" and insists on storing them in the snapshot.
A 1 gig file of zeros still takes several tens of megabytes after sis. hmm :) And roughly twice as much for a second copy of it.
3T for 3040 hmm. I have an existing volume that holds 2.5T, so that is
pushing it Why's that pushing it? sis should decrease the amt of used in that volume as you've already seen by 26%, maybe more after a few more passes.
I wonder why the max size scales with the system model, I haven't thought
of a good reason for this yet since you could easily have lots of 3T volumes. It scales with model because higher model >> more RAM and faster processors. Thus sis can address more storage. Yes, lots of 3T volumes, but that amt of 3T volumes will scale again by model platform...higher model, more RAM and processor, hence can address more storage (higher amount of disks).
I was wondering if you can get sis to shrink say 2.8T to 2.0, is the 2.0
what counts for the sis volume limit size? sis uses the size of the volume at time of turning sis on. If it's larger than your model will permit, it doesn't turn on and you'd have to shrink it. You can't grow the volume above the model limit if sis is turned on, you'd have to turn it off to grow.
Even so, would I run into trouble if trying to do a full volume restore
and the full data set is over 3.0 but would have shrunk? If you had to do a restore, it would be coming from either a sis'd volume elsewhere (say it's Snapmirrored), or a sis aware tape backup environment (could be mistaken there).
I noticed during my data copy, the snapshot reserve was blowing out big
time. This is normal. As you copy data in fresh, the snapshots (your backups) will put pointers to the blocks that have changed (the new data flowing in), All the data in essence has changed. If you're copying fresh data to the storage, may as well turn snapshots off until all the data is copied in. It won't be sis and snapshots contending here for same blocks...sis and snapshot's won't paper-rock-scissors for the same blocks as they both, in essence, treat blocks the same way (sis is highly based on snapshot foundation when you break it down to the 4k block level).
hth,
Best regards, ~~~~~~~~~~~~~~~~~~~~~ Kevin Parker http://theparkerz.com ~~~~~~~~~~~~~~~~~~~~~
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Adam McDougall Sent: Wednesday, March 26, 2008 10:08 PM To: toasters@mathworks.com Subject: Some thoughts and questions on A-sis
I started testing sis on one of our 3040's last weekend. Sorry if theres more extensive information on NOW, but the things I've found so far were fairly rudimentry. Some impressions and questions:
I found out quickly that my 5T test volume was too big. Looked up the limits because I had no idea there were volume size limits. 3T for 3040 hmm. I have an existing volume that holds 2.5T, so that is pushing it, and while I would like to split it up, it won't happen overnight. I'm not desperate to use sis on it, although I am almost done copying it and so far have realized 26% savings from dedupe for testing. I wonder why the max size scales with the system model, I haven't thought of a good reason for this yet since you could easily have lots of 3T volumes. I was wondering if you can get sis to shrink say 2.8T to 2.0, is the 2.0 what counts for the sis volume limit size? Even so, would I run into trouble if trying to do a full volume restore and the full data set is over 3.0 but would have shrunk? Once I write over 3T into a volume, it sounds like I cannot run sis on it. I'm not too worried about having to do a full restore of a volume from tape, but I don't want to limit my future options if the short term payoff isn't worth it. I'm pretty sure I'll use it on all or most of all of my other volumes since they are much smaller.
I noticed during my data copy, the snapshot reserve was blowing out big time. It seemed like almost any savings from sis went into snapshots for some reason. 100, 200, 400% full, and it started trimming itself back after a while. Since this was a non-production copy, I didn't care, and deleted those snapshots this morning, but I thought that was rather odd, and couldn't think what the snapshots would have to offer since if the data was unmodified at the filesystem level, the snapshot should contain the same data. While not a problem for an initial copy, I would expect the same thing to happen when sis runs during production, and although it wouldn't be as large and would eventually flush out, why am I expending space to store duplicate copies of data that I asked it to deduplicate? :) Maybe its just because wafl identifies them as "changed blocks" and insists on storing them in the snapshot.
A 1 gig file of zeros still takes several tens of megabytes after sis. hmm :) And roughly twice as much for a second copy of it.
FWIW, I'm pretty sure that tape backups are not SIS aware (none are that I'm aware of at least). When you write the data to tape via NDMP as a dump, then you're going to get _all_ data, and it will re-dupe it as each of the flexvol data blocks are accessed (remember, the de-dupe points multiple flex-vol blocks to a single aggregate block). I would think that snapmirror to tape would be an exception to this, but SMT isn't really a tape backup per se compared to regular DUMP (in some ways it's better, in some ways it's worse).
Glenn
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Kevin Parker Sent: Wednesday, March 26, 2008 11:56 PM To: 'Adam McDougall'; toasters@mathworks.com Subject: RE: Some thoughts and questions on A-sis
3T for 3040 hmm. I have an existing volume that holds 2.5T, so that
is pushing it Why's that pushing it? sis should decrease the amt of used in that volume as you've already seen by 26%, maybe more after a few more passes.
I wonder why the max size scales with the system model, I haven't
thought of a good reason for this yet since you could easily have lots of 3T volumes. It scales with model because higher model >> more RAM and faster processors. Thus sis can address more storage. Yes, lots of 3T volumes, but that amt of 3T volumes will scale again by model platform...higher model, more RAM and processor, hence can address more storage (higher amount of disks).
I was wondering if you can get sis to shrink say 2.8T to 2.0, is the
2.0 what counts for the sis volume limit size? sis uses the size of the volume at time of turning sis on. If it's larger than your model will permit, it doesn't turn on and you'd have to shrink it. You can't grow the volume above the model limit if sis is turned on, you'd have to turn it off to grow.
Even so, would I run into trouble if trying to do a full volume
restore and the full data set is over 3.0 but would have shrunk? If you had to do a restore, it would be coming from either a sis'd volume elsewhere (say it's Snapmirrored), or a sis aware tape backup environment (could be mistaken there).
I noticed during my data copy, the snapshot reserve was blowing out
big time. This is normal. As you copy data in fresh, the snapshots (your backups) will put pointers to the blocks that have changed (the new data flowing in), All the data in essence has changed. If you're copying fresh data to the storage, may as well turn snapshots off until all the data is copied in. It won't be sis and snapshots contending here for same blocks...sis and snapshot's won't paper-rock-scissors for the same blocks as they both, in essence, treat blocks the same way (sis is highly based on snapshot foundation when you break it down to the 4k block level).
hth,
Best regards, ~~~~~~~~~~~~~~~~~~~~~ Kevin Parker http://theparkerz.com ~~~~~~~~~~~~~~~~~~~~~
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Adam McDougall Sent: Wednesday, March 26, 2008 10:08 PM To: toasters@mathworks.com Subject: Some thoughts and questions on A-sis
I started testing sis on one of our 3040's last weekend. Sorry if theres more extensive information on NOW, but the things I've found so far were fairly rudimentry. Some impressions and questions:
I found out quickly that my 5T test volume was too big. Looked up the limits because I had no idea there were volume size limits. 3T for 3040 hmm. I have an existing volume that holds 2.5T, so that is pushing it, and while I would like to split it up, it won't happen overnight. I'm not desperate to use sis on it, although I am almost done copying it and so far have realized 26% savings from dedupe for testing. I wonder why the max size scales with the system model, I haven't thought of a good reason for this yet since you could easily have lots of 3T volumes. I was wondering if you can get sis to shrink say 2.8T to 2.0, is the 2.0 what counts for the sis volume limit size? Even so, would I run into trouble if trying to do a full volume restore and the full data set is over 3.0 but would have shrunk? Once I write over 3T into a volume, it sounds like I cannot run sis on it. I'm not too worried about having to do a full restore of a volume from tape, but I don't want to limit my future options if the short term payoff isn't worth it. I'm pretty sure I'll use it on all or most of all of my other volumes since they are much smaller.
I noticed during my data copy, the snapshot reserve was blowing out big time. It seemed like almost any savings from sis went into snapshots for some reason. 100, 200, 400% full, and it started trimming itself back after a while. Since this was a non-production copy, I didn't care, and deleted those snapshots this morning, but I thought that was rather odd, and couldn't think what the snapshots would have to offer since if the data was unmodified at the filesystem level, the snapshot should contain the same data. While not a problem for an initial copy, I would expect the same thing to happen when sis runs during production, and although it wouldn't be as large and would eventually flush out, why am I expending space to store duplicate copies of data that I asked it to deduplicate? :) Maybe its just because wafl identifies them as "changed blocks" and insists on storing them in the snapshot.
A 1 gig file of zeros still takes several tens of megabytes after sis. hmm :) And roughly twice as much for a second copy of it.
I've been creating my volumes =< 2TB _before_ I run SIS, to ensure that SIS will never complain about working on a volume larger than it can handle, including tape-restored volumes. For Windows file servers, I'm seeing ~30% space savings. For VMWare servers running 40ish similar Windows images, I'm getting ~90% space savings. Rough testing has indicated that I'm taking a 7-9% performance hit.
Your snapshots will definitely grow during the SIS runs, so I would look at turning snapshots off during that initial 'sis -s /vol/' process.
FWIW, I'm doing this with several LUNs per volume over FCP on a 3040.
Daniel
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Adam McDougall Sent: Wednesday, March 26, 2008 9:08 PM To: toasters@mathworks.com Subject: Some thoughts and questions on A-sis
I started testing sis on one of our 3040's last weekend. Sorry if theres more extensive information on NOW, but the things I've found so far were fairly rudimentry. Some impressions and questions:
I found out quickly that my 5T test volume was too big. Looked up the limits because I had no idea there were volume size limits. 3T for 3040
hmm. I have an existing volume that holds 2.5T, so that is pushing it, and while I would like to split it up, it won't happen overnight. I'm not desperate to use sis on it, although I am almost done copying it and
so far have realized 26% savings from dedupe for testing. I wonder why the max size scales with the system model, I haven't thought of a good reason for this yet since you could easily have lots of 3T volumes. I was wondering if you can get sis to shrink say 2.8T to 2.0, is the 2.0 what counts for the sis volume limit size? Even so, would I run into trouble if trying to do a full volume restore and the full data set is over 3.0 but would have shrunk? Once I write over 3T into a volume, it sounds like I cannot run sis on it. I'm not too worried about having to
do a full restore of a volume from tape, but I don't want to limit my future options if the short term payoff isn't worth it. I'm pretty sure
I'll use it on all or most of all of my other volumes since they are much smaller.
I noticed during my data copy, the snapshot reserve was blowing out big time. It seemed like almost any savings from sis went into snapshots for some reason. 100, 200, 400% full, and it started trimming itself back after a while. Since this was a non-production copy, I didn't care, and deleted those snapshots this morning, but I thought that was rather odd, and couldn't think what the snapshots would have to offer since if the data was unmodified at the filesystem level, the snapshot should contain the same data. While not a problem for an initial copy, I would expect the same thing to happen when sis runs during production,
and although it wouldn't be as large and would eventually flush out, why
am I expending space to store duplicate copies of data that I asked it to deduplicate? :) Maybe its just because wafl identifies them as "changed blocks" and insists on storing them in the snapshot.
A 1 gig file of zeros still takes several tens of megabytes after sis. hmm :) And roughly twice as much for a second copy of it.
______________________________________________________________________ This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
So, if A-SIS is limited to volumes of under 3.5Tb, what do those of us with 10Tb volumes do?
John
The size limit varies on what filer you are on.
The limit is 10TB on the 6030 and 14TB on the 6070.
The table of the various limits is here: http://now.netapp.com/NOW/knowledge/docs/ontap/rel723/html/ontap/onlineb k/6asis.htm
John
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of John Stoffel Sent: Thursday, March 27, 2008 1:29 PM To: Daniel Keisling Cc: toasters@mathworks.com Subject: RE: Some thoughts and questions on A-sis
So, if A-SIS is limited to volumes of under 3.5Tb, what do those of us with 10Tb volumes do?
John
-
Adam McDougall -
Clear, John -
Daniel Keisling -
Glenn Walker -
John Stoffel -
Kevin Parker