I used autogrow on GX, it was one feature that was rock-solid. What version of GX are you on? GX itself was quite solid with the last version released. Are you the last person still using it :) ? I thought i'd have been!On Thu, Dec 10, 2015 at 2:07 PM, Mike Thompson <mike.thompson@gmail.com> wrote:Thanks everyoneI'll have to experiment with the autogrow stuff - I recall we were steered away from that by NetApp themselves saying it wasn't all that stable in Ontap 8.0, but never did fool with it. I will give it a shot though.Thanks SebastianI as well figured it was something like that - thanks for the link - makes sense.On Thu, Dec 10, 2015 at 8:13 AM, Sebastian Goetze <spgoetze@gmail.com> wrote:I didn't do the math, but just an idea:
ONTAP calculates an average file size of 32K to determine the number of inodes per volume. Every inode takes 192 Bytes from the INODEFILE. So, if you take 2TB / 32KB * 192B = 12GB... (actually I *did* do the math right now, I was curious...)
It seems this is enforced on *newer* aggregates...
According to KB https://kb.netapp.com/support/index?page=content&id=3011432 you can't go below the VolSize/32K limit.
How about provisioning the volumes small (and thin) and letting them *AutoGrow* to 2TB?
That way, they're not using up space (not even for inodes), yet are able to contain up to 2TB...
Just my 2c
Sebastian
On 12/10/2015 8:43 AM, Mike Thompson wrote:
recently built aggregate under Ontap8:for example:the aggregates that were originally created as GX aggrs, have identically matching 'allocated' and 'used' values in 'aggr show_space'what's strange is this is happening on some filers/aggregates but not others.it appears to be related to the size of the volume. we thin-provision all volumes by setting them to a large size and set space guarantee to none.if I move vols between affected and unaffected filers/aggrs, the allocated vs used normalizes for the volumes on the unaffected node, then re-inflate when moved back to the original node/aggrsample vol and aggr details belowcan't upgrade, not on support, we have about 1PB in production across two clusters. I am seeing this effect on both clusters, but not on all filers, which is strange.
Volume Allocated Used
vol created on problem aggr v164402 11932348KB 1924KB
after move to unaffected aggr v164402 2872KB 2872KB
after move back to orig aggr v164402 12005552KB 75284KB
resized to 1TB, moved to another aggr, and back to orig aggr
v164402 6003080KB 37972KB
vols sized at 2TB end up being allocated 12GB of space when created. vols sized to 1TB end up with 6GB allocated when created, even though they are completely empty.
these clusters originated running Ontap GX, and then were upgraded to Ontap8 c-mode some time ago.
the aggregates that seem to be experiencing this allocation 'inflation' seem to be those that were created after the cluster was upgraded to Ontap8.
Aggregate Allocated Used Avail
Total space 18526684036KB 13662472732KB 1233809368KB <- massive difference!
aggregate originated under Ontap GX:
Aggregate Allocated Used Avail
Total space 32035928920KB 32035928920KB 1404570796KB <- identical, which is what i would expect
maybe this allocation scheme changed at an aggregate level under Ontap 8? perhaps it's expected behavior?
things do seem to normalize as the volumes begin to fill up though, so I believe that this space is not truly gone permanently, but it certainly appears to be not available for use, since tons of space is allocated to volumes that have very little data in them, and we have a LOT of volumes in these clusters.
it's definitely making it look like we are missing a substantial percentage of our disk space, when trying to reconcile the sum of data used when tallying up volume size, and comparing it to the aggregate used/remaining sizes.
example volume and affected aggregate details:
bc-gx-4b::*> vol show v164346 -instance
(volume show)
Virtual Server Name: bc
Volume Name: v164346
Aggregate Name: gx4b_1
Volume Size: 2TB
Name Ordinal: base
Volume Data Set ID: 4041125
Volume Master Data Set ID: 2151509138
Volume State: online
Volume Type: RW
Volume Style: flex
Volume Ownership: cluster
Export Policy: default
User ID: jobsys
Group ID: cgi
Security Style: unix
Unix Permissions: ---rwxr-x--x
Junction Path: /bc/shows/ID2/DFO/0820
Junction Path Source: RW_volume
Junction Active: true
Parent Volume: v164232
Virtual Server Root Volume: false
Comment:
Available Size: 1.15TB
Total Size: 2TB
Used Size: 146.7MB
Used Percentage: 42%
Autosize Enabled (for flexvols only): false
Maximum Autosize (for flexvols only): 2.40TB
Autosize Increment (for flexvols only): 102.4GB
Total Files (for user-visible data): 31876689
Files Used (for user-visible data): 132
Maximum Directory Size: 100MB
Space Guarantee Style: none
Space Guarantee In Effect: true
Minimum Read Ahead: false
Access Time Update Enabled: true
Snapshot Directory Access Enabled: true
Percent of Space Reserved for Snapshots: 0%
Used Percent of Snapshot Reserve: 0%
Snapshot Policy: daily
Creation Time: Tue Dec 08 11:22:58 2015
Language: C
Striped Data Volume Count: -
Striped Data Volume Stripe Width: 0.00B
Current Striping Epoch: -
One data-volume per member aggregate: -
Concurrency Level: -
Optimization Policy: -
Clone Volume: false
Anti-Virus On-Access Policy: default
UUID of the volume: 17fa4c6d-9de1-11e5-a888-123478563412
Striped Volume Format: -
Load Sharing Source Volume: -
Move Target Volume: false
Maximum Write Alloc Blocks: 0
Inconsistency in the file system: false
bc-gx-4b::*> aggr show -aggregate
far4a_1 gx1a_1 gx1a_2 gx1b_1 gx2a_1 gx2b_1 gx3a_1 gx4b_1
near1b_1 near3b_1 root_1a root_1b root_2a root_2b root_3a root_3b
root_4a root_4b slow2a_1 systems
bc-gx-4b::*> aggr show -aggregate gx4b_1
Aggregate: gx4b_1
UUID: c624f85e-96d3-11e3-a6ce-00a0980bb25a
Size: 18.40TB
Used Size: 17.25TB
Used Percentage: 94%
Available Size: 1.15TB
State: online
Nodes: bc-gx-4b
Number Of Disks: 63
Disks: bc-gx-4b:0a.64, bc-gx-4b:0e.80,
...
bc-gx-4b:0a.45
Number Of Volumes: 411
Plexes: /gx4b_1/plex0(online)
RAID Groups: /gx4b_1/plex0/rg0,
/gx4b_1/plex0/rg1,
/gx4b_1/plex0/rg2
Raid Type: raid_dp
Max RAID Size: 21
RAID Status: raid_dp
Checksum Enabled: true
Checksum Status: active
Checksum Style: block
Inconsistent: false
Ignore Inconsistent: off
Block Checksum Protection: on
Zoned Checksum Protection: -
Automatic Snapshot Deletion: on
Enable Thorough Scrub: off
Volume Style: flex
Volume Types: flex
Has Mroot Volume: false
Has Partner Node Mroot Volume: false
Is root: false
Wafliron Status: -
Percent Blocks Scanned: -
Last Start Error Number: -
Last Start Error Info: -
Aggregate Type: aggr
Number of Quiesced Volumes: -
Number of Volumes not Online: -
Number of LS Mirror Destination Volumes: -
Number of DP Mirror Destination Volumes: -
Number of Move Mirror Destination Volumes: -
Number of DP qtree Mirror Destination Volumes: -
HA Policy: sfo
Block Type: 64-bit
On Wed, Dec 9, 2015 at 12:50 PM, John Stoffel <john@stoffel.org> wrote:
Can you post the details of one of these volumes? And of the
aggregate you have them in? It smells like there's some sort of
minimum volume size setting somewhere.
Or maybe there's an aggregate level snapshot sitting around?
Can you upgrade? You're in cluster mode, so hopefully it shoudln't be
too hard to move to 8.1, then 8.2 and onto 8.3, since there's lots of
nice bug fixes.
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters