For anyone interested this was (is) being cause by SIS running on the effected RGs, I'm moving a 1/2 TB a day or so to the new system so it's got a lot to crunch right now. I'll send another message out in a week or two and let folks know how it is acting once things calm down.
So far all NFS for our moderately sized ESX environment is running very well. We've got 140 VMs moved over so far and things are running very smoothly.
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Page, Jeremy Sent: Thursday, June 05, 2008 8:38 PM To: Blake Golliher; Blackmor, Chris Cc: Hadrian.Baron@vegas.com; toasters@mathworks.com Subject: RE: How to identify a hot disk
I don't understand how the fragmentation occurred in the first place, this is a brand new filer, we've not even had any snapshots age out yet. All the disks where added to the aggr at the same time, lots of free space.
The only thing odd is that we did have ASIS run, I wonder if the fragmentation is from all the stuff getting de-duped (it's a bunch of VMs)
-----Original Message----- From: Blake Golliher [mailto:thelastman@gmail.com] Sent: Thursday, June 05, 2008 5:17 PM To: Blackmor, Chris Cc: Hadrian.Baron@vegas.com; Page, Jeremy; toasters@mathworks.com Subject: Re: How to identify a hot disk
It'll depend on how busy your disks are during prime time. Safety first people!
On Thu, Jun 5, 2008 at 1:38 PM, Blackmor, Chris Chris.Blackmor@amd.com wrote:
My understanding of reallocate is that it will put a definite load on
your
filer and shouldn't be run during prime time.
----- Original Message ----- From: owner-toasters@mathworks.com owner-toasters@mathworks.com To: 'Page, Jeremy' jeremy.page@gilbarco.com; toasters@mathworks.com toasters@mathworks.com Sent: Thu Jun 05 10:57:12 2008 Subject: RE: How to identify a hot disk
It's good you are at 50% aggr usage, as you'll need 50% free space in
each
volume you run the reallocate on. I think running the reallocate is
the
best first step as it is fairly un-intrusive and you can run it during
the
day unless you are hammering the filer constantly. When we run it we
use
the parameter -f to force reallocation without caring how well it is
already
laid out. Not sure about your question on A-SIS.
HTH,
- Hadrian
From: owner-toasters@mathworks.com
[mailto:owner-toasters@mathworks.com] On
Behalf Of Page, Jeremy Sent: Thursday, June 05, 2008 7:13 AM To: toasters@mathworks.com Subject: RE: How to identify a hot disk
Thu Jun 5 10:06:02 EDT [gvr-array02: wafl.scan.layout.advise:info]:
WAFL
layout ratio for volume nfs2 is 4.01. A ratio of 1 is optimal. Based
on your
free space, 1.42 is expected.
Would you say I need to do a reallocate? I'm not sure why this is so fragmented, this file system has never been more then 50% full, could
A-SIS
have something to do with it?
Jeremy M. Page____________________
Systems Architect
- email:Jeremy.Page@gilbarco.com - ( phone: 336.547.5399 - 6 fax:
336.547.5163 - ( cell: 336.601.7274
From: Uddhav Regmi [mailto:uddhav.regmi@worldnet.att.net] Sent: Thursday, June 05, 2008 9:38 AM To: Page, Jeremy; toasters@mathworks.com Subject: RE: How to identify a hot disk
hmmm
very interesting....
looks like those are max out
do wafl scan measure layout and see where you stand....
if needed do reallocate.....
I have seen hundreds of cases where it helped a lot
-uddhav
From: owner-toasters@mathworks.com
[mailto:owner-toasters@mathworks.com] On
Behalf Of Page, Jeremy Sent: Thursday, June 05, 2008 8:27 AM To: toasters@mathworks.com Subject: Re: How to identify a hot disk
Not sure why but I have two disks that are maxed out while the rest
are far
lower utilization. What would cause this, the raid groups where
created all
at the same time, there are 10 disks per raid group and 3 groups in
the
aggragate.
/aggr0/plex0/rg0:
1c.16 2 0.94 0.18 1.00 42250 0.49 12.18 1396
0.27
11.83 521 0.00 .... . 0.00 .... .
1c.32 6 2.56 0.49 1.00 104545 1.80 4.18 1323 0.27 11.83 634 0.00 .... . 0.00 .... .
1c.48 98 116.05 114.20 1.62 18481 1.53 4.24 1743
0.31
11.00 2649 0.00 .... . 0.00 .... .
1c.17 41 55.21 54.76 2.17 5697 0.27 20.17 1537
0.18
16.25 600 0.00 .... . 0.00 .... .
1c.33 49 72.07 71.62 1.84 4760 0.27 20.67 871
0.18
16.75 1239 0.00 .... . 0.00 .... .
1c.49 45 61.42 60.97 2.18 4047 0.22 23.00 913
0.22
14.40 931 0.00 .... . 0.00 .... .
1c.64 5 117.40 116.86 1.61 307 0.36 15.25 1336
0.18
18.00 319 0.00 .... . 0.00 .... .
1c.65 4 71.76 71.17 1.83 292 0.27 20.67 1298
0.31
10.43 370 0.00 .... . 0.00 .... .
1c.80 4 49.50 49.05 2.36 333 0.22 22.80 1333
0.22
14.60 712 0.00 .... . 0.00 .... .
1c.81 5 95.14 94.69 1.71 292 0.22 22.80 1325
0.22
14.60 548 0.00 .... . 0.00 .... .
/aggr0/plex0/rg1:
1c.66 1 0.67 0.00 .... . 0.31 19.29 1311
0.36
10.50 238 0.00 .... . 0.00 .... .
1c.83 1 0.67 0.00 .... . 0.31 19.29 1415
0.36
10.50 190 0.00 .... . 0.00 .... .
1c.82 4 48.60 48.24 2.21 320 0.22 22.80 1553
0.13
21.33 234 0.00 .... . 0.00 .... .
1c.19 51 67.89 67.44 1.92 5315 0.22 22.80 1281
0.22
13.20 788 0.00 .... . 0.00 .... .
1c.18 55 72.75 72.34 1.90 4996 0.22 23.00 1122
0.18
16.00 1109 0.00 .... . 0.00 .... .
1c.35 30 36.10 35.52 2.67 3190 0.31 15.86 1802
0.27
11.33 588 0.00 .... . 0.00 .... .
1c.34 41 52.97 52.43 2.04 4207 0.31 16.29 1570
0.22
13.60 750 0.00 .... . 0.00 .... .
1c.51 100 119.82 119.46 1.57 25873 0.22 22.80 1588
0.13
21.33 2313 0.00 .... . 0.00 .... .
1c.50 59 72.03 71.44 1.83 7750 0.27 19.33 1233
0.31
10.86 1289 0.00 .... . 0.00 .... .
1c.67 4 94.60 94.15 1.68 279 0.18 24.50 1071
0.27
12.33 338 0.00 .... . 0.00 .... .
/aggr0/plex0/rg2:
1c.85 1 0.94 0.00 .... . 0.54 12.00 1806
0.40
10.11 538 0.00 .... . 0.00 .... .
1c.84 1 0.94 0.00 .... . 0.54 12.00 1729
0.40
10.11 495 0.00 .... . 0.00 .... .
1c.21 47 67.40 66.90 1.86 4701 0.27 19.17 1452
0.22
14.20 845 0.00 .... . 0.00 .... .
1c.20 56 73.56 73.02 1.79 4866 0.18 25.50 1039 0.36 9.63 870 0.00 .... . 0.00 .... .
1c.37 51 68.79 68.21 1.76 6072 0.27 19.17 1174
0.31
11.43 988 0.00 .... . 0.00 .... .
1c.36 42 50.85 50.18 2.31 3807 0.40 12.78 1852
0.27
13.33 800 0.00 .... . 0.00 .... .
1c.53 59 75.85 75.18 1.86 5024 0.36 14.25 2237
0.31
10.43 493 0.00 .... . 0.00 .... .
1c.52 18 21.27 20.77 3.83 2205 0.27 19.17 1496
0.22
14.20 465 0.00 .... . 0.00 .... .
1c.69 5 71.76 71.00 1.96 296 0.40 12.78 2087 0.36 9.63 610 0.00 .... . 0.00 .... .
1c.68 5 71.58 71.13 1.84 352 0.22 23.60 1314
0.22
14.80 514 0.00 .... . 0.00 .... .
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.