Is it stable? I mean 8.1.3 is just in GA for now...
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atmailto:ag@anexia.at Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Peter-Paul Witta [mailto:paul.witta@CUBiT.at] Gesendet: Samstag, 19. Oktober 2013 00:30 An: Alexander Griesser; Sebastian Goetze Cc: toasters@teaparty.net Betreff: Re: Dedup killing performance (again)
I'd recommend 813p2 immediately...
lg paul witta
Am 19.10.13 00:19, schrieb Alexander Griesser: Yah, just found it while googling I guess, must be bug id #657692 :-/
Anyhow, I'm now stuck with this:
sis status
Path State Status Progress /vol/CIFS_SHARES Enabled Active Fingerprint is being recovered from aggregate (6% complete)
It's doing that since quite some time now, although it now actually jumped from 0% to 6%, so it at least is still alive.
This bug is first fixed in 8.1.2P4 as it seems, so I might need to upgrade ASAP; for now, I'll check if it at least finishes this process now, my volume is filling up quickly, but there's still some aggregate space left so I'll just increase the volume size before it goes full. I'll go above the magic 80% fill of an aggregate then, but better that then having the volume go full.
I'll see how it goes and if it troubles me too much, I hope there is a way to abort sis in that state.
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atmailto:ag@anexia.at Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Sebastian Goetze [mailto:spgoetze@gmail.com] Gesendet: Samstag, 19. Oktober 2013 00:16 An: Alexander Griesser Betreff: Re: Dedup killing performance (again)
Isn't that the Dedupe bug again, that was fixed in 8.1.3? (since like 8.1.2P3 or P4)...
There was a thread about it some time ago...
Anyway, I'm off to bed, just wanted to point you in the right direction.
BFN
On 10/18/2013 11:52 PM, Alexander Griesser wrote: Hi there,
I am just trying to dedupe some volumes on one of our filers and the first volume (2.4TB data, CIFS share) deduplicates since about 4 hours now. Sysstat -x 1 shows:
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk OTHER FCP iSCSI FCP kB/s iSCSI kB/s in out read write read write age hit time ty util in out in out 58% 115 0 0 116 2239 3635 281593 32 0 0 0s 98% 0% - 100% 0 0 1 0 0 0 0 63% 162 0 0 169 1682 4149 310829 0 0 0 0s 99% 0% - 100% 7 0 0 0 0 0 0 50% 381 0 0 384 11793 11325 182100 0 0 0 0s 96% 0% - 100% 0 0 3 0 0 0 0 55% 412 0 0 416 6487 6040 223036 24 0 0 0s 97% 0% - 100% 0 0 4 0 0 37 0 51% 19 0 0 19 185 9 239388 0 0 0 2 98% 0% - 100% 0 0 0 0 0 0 0 51% 77 0 0 80 348 906 273880 8 0 0 2 99% 0% - 100% 0 0 3 0 0 0 0 60% 48 0 0 49 174 410 196179 271 0 0 2 98% 35% Tn 100% 0 0 1 0 0 0 0 45% 102 0 0 110 337 337 144387 21222 0 0 2 96% 100% :f 100% 8 0 0 0 0 0 0 71% 143 0 0 149 258 2814 109728 24268 0 0 2 85% 100% :f 86% 0 0 6 0 0 25 0 53% 64 0 0 65 203 1368 236620 14152 0 0 2 98% 100% :f 100% 0 0 1 0 0 0 0 49% 96 0 0 96 319 1317 185552 19748 0 0 2 96% 100% :f 100% 0 0 0 0 0 0 0 35% 109 0 0 117 381 1962 62613 18010 0 0 2 88% 100% :f 100% 5 0 3 0 0 0 0 55% 38 0 0 39 159 1168 239408 1980 0 0 2 97% 100% :f 100% 0 0 1 0 0 0 0 57% 177 0 0 179 308 6692 136568 520 0 0 2 92% 82% : 99% 0 0 2 0 0 37 0 39% 256 0 0 271 403 12647 117284 0 0 0 2 93% 0% - 97% 12 0 3 0 0 0 0 40% 232 0 0 473 808 7506 162368 32 0 0 2 96% 0% - 100% 240 0 1 0 0 0 0 38% 86 0 0 86 246 3413 160092 0 0 0 2 96% 0% - 100% 0 0 0 0 0 0 0 47% 113 0 0 117 314 1738 199516 0 0 0 2 96% 0% - 100% 1 0 3 0 0 0 0 59% 116 0 0 120 476 2751 288048 24 0 0 0s 99% 0% - 100% 0 0 4 0 0 24 0 45% 230 0 0 230 2879 6547 185787 120 0 0 0s 98% 0% - 100% 0 0 0 0 0 0 0
version
NetApp Release 8.1.2P2 7-Mode: Wed Feb 20 19:58:27 PST 2013
I'm aware that deduplication _SHOULD_ be a background process hugging up as much resources as it can but when host i/o is needed, it will be preferred over deduplication. There are no other tasks running at the same time (snapvault, etc.), and the virtual machinse on this system are _VERY_ slow - f.ex. logging in via RDP on one of the virtual machines now took > 1 minute for the login window to appear.
This is a FAS2240 with 24x600 SAS 10k and basically my question now is: Is there any way to tell a-sis to really run with lower priority or is this a bug in the OnTap version I'm running? I've used a-sis a lot in the past and never had performance issues like this when it was running...
I'm thankful for any ideas how to track down this issue or improve performance while a-sis is running.
Thanks,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atmailto:ag@anexia.at Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
_______________________________________________
Toasters mailing list
Toasters@teaparty.netmailto:Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.netmailto:Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
--
kind regards, +43-664-4542287
Ing. Peter-Paul Witta +43-1-7189880-0
Chief Technology Officer www.CUBiT.athttp://www.CUBiT.at
I would agree!! I upgraded to 8.1.3p2 on several arrays over the past month and there hasn't been any fall out.
-Mark On Oct 18, 2013 4:39 PM, "Alexander Griesser" ag@anexia.at wrote:
Is it stable? I mean 8.1.3 is just in GA for now…****
*Alexander Griesser*
System-Administrator****
ANEXIA Internetdienstleistungs GmbH****
Telefon: +43-5-0556-320****
Telefax: +43-5-0556-500****
E-Mail: ag@anexia.at****
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt***
Geschäftsführer: Alexander Windbichler****
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601****
*Von:* Peter-Paul Witta [mailto:paul.witta@CUBiT.at] *Gesendet:* Samstag, 19. Oktober 2013 00:30 *An:* Alexander Griesser; Sebastian Goetze *Cc:* toasters@teaparty.net *Betreff:* Re: Dedup killing performance (again)****
I'd recommend 813p2 immediately...
lg paul witta
Am 19.10.13 00:19, schrieb Alexander Griesser:****
Yah, just found it while googling I guess, must be bug id #657692 :-/****
Anyhow, I’m now stuck with this:****
sis status****
Path State Status Progress****
/vol/CIFS_SHARES Enabled Active Fingerprint is being recovered from aggregate (6% complete)****
It’s doing that since quite some time now, although it now actually jumped from 0% to 6%, so it at least is still alive.****
This bug is first fixed in 8.1.2P4 as it seems, so I might need to upgrade ASAP; for now, I’ll check if it at least finishes this process now, my volume is filling up quickly, but there’s still some aggregate space left so I’ll just increase the volume size before it goes full.****
I’ll go above the magic 80% fill of an aggregate then, but better that then having the volume go full.****
I’ll see how it goes and if it troubles me too much, I hope there is a way to abort sis in that state.****
*Alexander Griesser*****
System-Administrator****
ANEXIA Internetdienstleistungs GmbH****
Telefon: +43-5-0556-320****
Telefax: +43-5-0556-500****
E-Mail: ag@anexia.at****
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt***
Geschäftsführer: Alexander Windbichler****
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601****
*Von:* Sebastian Goetze [mailto:spgoetze@gmail.com spgoetze@gmail.com] *Gesendet:* Samstag, 19. Oktober 2013 00:16 *An:* Alexander Griesser *Betreff:* Re: Dedup killing performance (again)****
Isn't that the Dedupe bug again, that was fixed in 8.1.3? (since like 8.1.2P3 or P4)...
There was a thread about it some time ago...
Anyway, I'm off to bed, just wanted to point you in the right direction.
BFN
On 10/18/2013 11:52 PM, Alexander Griesser wrote:****
Hi there,****
I am just trying to dedupe some volumes on one of our filers and the first volume (2.4TB data, CIFS share) deduplicates since about 4 hours now.****
Sysstat –x 1 shows:****
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk OTHER FCP iSCSI FCP kB/s iSCSI kB/s****
in out read write readwrite age hit time ty util in out in out****
58% 115 0 0 116 2239 3635 281593 32 0 0 0s 98% 0% - 100% 0 0 1 0 0 0 0****
63% 162 0 0 169 1682 4149 310829 0 0 0 0s 99% 0% - 100% 7 0 0 0 0 0 0****
50% 381 0 0 384 11793 11325 182100 0 0 0 0s 96% 0% - 100% 0 0 3 0 0 0 0****
55% 412 0 0 416 6487 6040 223036 24 0 0 0s 97% 0% - 100% 0 0 4 0 0 37 0****
51% 19 0 0 19 185 9 239388 0 0 0 2 98% 0% - 100% 0 0 0 0 0 0 0****
51% 77 0 0 80 348 906 273880 8 0 0 2 99% 0% - 100% 0 0 3 0 0 0 0****
60% 48 0 0 49 174 410 196179 271 0 0 2 98% 35% Tn 100% 0 0 1 0 0 0 0****
45% 102 0 0 110 337 337 144387 21222 0 0 2 96% 100% :f 100% 8 0 0 0 0 0 0****
71% 143 0 0 149 258 2814 109728 24268 0 0 2 85% 100% :f 86% 0 0 6 0 0 25 0****
53% 64 0 0 65 203 1368 236620 14152 0 0 2 98% 100% :f 100% 0 0 1 0 0 0 0****
49% 96 0 0 96 319 1317 185552 19748 0 0 2 96% 100% :f 100% 0 0 0 0 0 0 0****
35% 109 0 0 117 381 1962 62613 18010 0 0 2 88% 100% :f 100% 5 0 3 0 0 0 0****
55% 38 0 0 39 159 1168 239408 1980 0 0 2 97% 100% :f 100% 0 0 1 0 0 0 0****
57% 177 0 0 179 308 6692 136568 520 0 0 2 92% 82% : 99% 0 0 2 0 0 37 0****
39% 256 0 0 271 403 12647 117284 0 0 0 2 93% 0% - 97% 12 0 3 0 0 0 0****
40% 232 0 0 473 808 7506 162368 32 0 0 2 96% 0% - 100% 240 0 1 0 0 0 0****
38% 86 0 0 86 246 3413 160092 0 0 0 2 96% 0% - 100% 0 0 0 0 0 0 0****
47% 113 0 0 117 314 1738 199516 0 0 0 2 96% 0% - 100% 1 0 3 0 0 0 0****
59% 116 0 0 120 476 2751 288048 24 0 0 0s 99% 0% - 100% 0 0 4 0 0 24 0****
45% 230 0 0 230 2879 6547 185787 120 0 0 0s 98% 0% - 100% 0 0 0 0 0 0 0****
version****
NetApp Release 8.1.2P2 7-Mode: Wed Feb 20 19:58:27 PST 2013****
I’m aware that deduplication _*SHOULD*_ be a background process hugging up as much resources as it can but when host i/o is needed, it will be preferred over deduplication.****
There are no other tasks running at the same time (snapvault, etc.), and the virtual machinse on this system are _*VERY*_ slow – f.ex. logging in via RDP on one of the virtual machines now took > 1 minute for the login window to appear.****
This is a FAS2240 with 24x600 SAS 10k and basically my question now is: Is there any way to tell a-sis to really run with lower priority or is this a bug in the OnTap version I’m running? I’ve used a-sis a lot in the past and never had performance issues like this when it was running…****
I’m thankful for any ideas how to track down this issue or improve performance while a-sis is running.****
Thanks,****
*Alexander Griesser*****
System-Administrator****
ANEXIA Internetdienstleistungs GmbH****
Telefon: +43-5-0556-320****
Telefax: +43-5-0556-500****
E-Mail: ag@anexia.at****
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt***
Geschäftsführer: Alexander Windbichler****
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601****
_______________________________________________****
Toasters mailing list****
Toasters@teaparty.net****
http://www.teaparty.net/mailman/listinfo/toasters****
_______________________________________________****
Toasters mailing list****
Toasters@teaparty.net****
http://www.teaparty.net/mailman/listinfo/toasters****
-- ****
kind regards, +43-664-4542287****
Ing. Peter-Paul Witta +43-1-7189880-0****
Chief Technology Officer www.CUBiT.at****
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I have 8.1.3p1 on a 3140 and a 3270 (7 mode) with no dedupe performance issues.
Sent from Yahoo! Mail on Android
Alright, I’m almost convinced to go for 8.1.3 now – will read through the changelogs tomorrow morning; right now, everything is fine again, fingerprint recovery has been finished a few minutes after my last mail regarding this issue and I’m now deleting some old snapshots to regain that space which should be fine for now.
I’ve disabled any future sis schedules for now until I know how I want to proceed with that situation.
Thanks for all your fast help as always ☺
Bye,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atmailto:ag@anexia.at Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Fred Grieco [mailto:fredgrieco@yahoo.com] Gesendet: Samstag, 19. Oktober 2013 02:11 An: Alexander Griesser; Mark Allen Cc: Peter-Paul Witta; toasters@teaparty.net Betreff: Re: AW: Dedup killing performance (again)
I have 8.1.3p1 on a 3140 and a 3270 (7 mode) with no dedupe performance issues.
Sent from Yahoo! Mail on Android
________________________________ From: Mark Allen <mmmjka@gmail.commailto:mmmjka@gmail.com>; To: Alexander Griesser <ag@anexia.atmailto:ag@anexia.at>; Cc: Peter-Paul Witta <paul.witta@cubit.atmailto:paul.witta@cubit.at>; <toasters@teaparty.netmailto:toasters@teaparty.net>; Subject: Re: AW: Dedup killing performance (again) Sent: Sat, Oct 19, 2013 12:00:06 AM
I would agree!! I upgraded to 8.1.3p2 on several arrays over the past month and there hasn't been any fall out.
-Mark On Oct 18, 2013 4:39 PM, "Alexander Griesser" <ag@anexia.atjavascript:return> wrote: Is it stable? I mean 8.1.3 is just in GA for now…
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Peter-Paul Witta [mailto:paul.witta@CUBiT.atjavascript:return] Gesendet: Samstag, 19. Oktober 2013 00:30 An: Alexander Griesser; Sebastian Goetze Cc: toasters@teaparty.netjavascript:return Betreff: Re: Dedup killing performance (again)
I'd recommend 813p2 immediately...
lg paul witta
Am 19.10.13 00:19, schrieb Alexander Griesser: Yah, just found it while googling I guess, must be bug id #657692 :-/
Anyhow, I’m now stuck with this:
sis status
Path State Status Progress /vol/CIFS_SHARES Enabled Active Fingerprint is being recovered from aggregate (6% complete)
It’s doing that since quite some time now, although it now actually jumped from 0% to 6%, so it at least is still alive.
This bug is first fixed in 8.1.2P4 as it seems, so I might need to upgrade ASAP; for now, I’ll check if it at least finishes this process now, my volume is filling up quickly, but there’s still some aggregate space left so I’ll just increase the volume size before it goes full. I’ll go above the magic 80% fill of an aggregate then, but better that then having the volume go full.
I’ll see how it goes and if it troubles me too much, I hope there is a way to abort sis in that state.
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Sebastian Goetze [mailto:spgoetze@gmail.comjavascript:return] Gesendet: Samstag, 19. Oktober 2013 00:16 An: Alexander Griesser Betreff: Re: Dedup killing performance (again)
Isn't that the Dedupe bug again, that was fixed in 8.1.3? (since like 8.1.2P3 or P4)...
There was a thread about it some time ago...
Anyway, I'm off to bed, just wanted to point you in the right direction.
BFN
On 10/18/2013 11:52 PM, Alexander Griesser wrote: Hi there,
I am just trying to dedupe some volumes on one of our filers and the first volume (2.4TB data, CIFS share) deduplicates since about 4 hours now. Sysstat –x 1 shows:
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk OTHER FCP iSCSI FCP kB/s iSCSI kB/s in out read write read write age hit time ty util in out in out 58% 115 0 0 116 2239 3635 281593 32 0 0 0s 98% 0% - 100% 0 0 1 0 0 0 0 63% 162 0 0 169 1682 4149 310829 0 0 0 0s 99% 0% - 100% 7 0 0 0 0 0 0 50% 381 0 0 384 11793 11325 182100 0 0 0 0s 96% 0% - 100% 0 0 3 0 0 0 0 55% 412 0 0 416 6487 6040 223036 24 0 0 0s 97% 0% - 100% 0 0 4 0 0 37 0 51% 19 0 0 19 185 9 239388 0 0 0 2 98% 0% - 100% 0 0 0 0 0 0 0 51% 77 0 0 80 348 906 273880 8 0 0 2 99% 0% - 100% 0 0 3 0 0 0 0 60% 48 0 0 49 174 410 196179 271 0 0 2 98% 35% Tn 100% 0 0 1 0 0 0 0 45% 102 0 0 110 337 337 144387 21222 0 0 2 96% 100% :f 100% 8 0 0 0 0 0 0 71% 143 0 0 149 258 2814 109728 24268 0 0 2 85% 100% :f 86% 0 0 6 0 0 25 0 53% 64 0 0 65 203 1368 236620 14152 0 0 2 98% 100% :f 100% 0 0 1 0 0 0 0 49% 96 0 0 96 319 1317 185552 19748 0 0 2 96% 100% :f 100% 0 0 0 0 0 0 0 35% 109 0 0 117 381 1962 62613 18010 0 0 2 88% 100% :f 100% 5 0 3 0 0 0 0 55% 38 0 0 39 159 1168 239408 1980 0 0 2 97% 100% :f 100% 0 0 1 0 0 0 0 57% 177 0 0 179 308 6692 136568 520 0 0 2 92% 82% : 99% 0 0 2 0 0 37 0 39% 256 0 0 271 403 12647 117284 0 0 0 2 93% 0% - 97% 12 0 3 0 0 0 0 40% 232 0 0 473 808 7506 162368 32 0 0 2 96% 0% - 100% 240 0 1 0 0 0 0 38% 86 0 0 86 246 3413 160092 0 0 0 2 96% 0% - 100% 0 0 0 0 0 0 0 47% 113 0 0 117 314 1738 199516 0 0 0 2 96% 0% - 100% 1 0 3 0 0 0 0 59% 116 0 0 120 476 2751 288048 24 0 0 0s 99% 0% - 100% 0 0 4 0 0 24 0 45% 230 0 0 230 2879 6547 185787 120 0 0 0s 98% 0% - 100% 0 0 0 0 0 0 0
version
NetApp Release 8.1.2P2 7-Mode: Wed Feb 20 19:58:27 PST 2013
I’m aware that deduplication _SHOULD_ be a background process hugging up as much resources as it can but when host i/o is needed, it will be preferred over deduplication. There are no other tasks running at the same time (snapvault, etc.), and the virtual machinse on this system are _VERY_ slow – f.ex. logging in via RDP on one of the virtual machines now took > 1 minute for the login window to appear.
This is a FAS2240 with 24x600 SAS 10k and basically my question now is: Is there any way to tell a-sis to really run with lower priority or is this a bug in the OnTap version I’m running? I’ve used a-sis a lot in the past and never had performance issues like this when it was running…
I’m thankful for any ideas how to track down this issue or improve performance while a-sis is running.
Thanks,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
_______________________________________________
Toasters mailing list
Toasters@teaparty.netjavascript:return
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.netjavascript:return
http://www.teaparty.net/mailman/listinfo/toasters
--
kind regards, +43-664-4542287
Ing. Peter-Paul Witta +43-1-7189880-0
Chief Technology Officer www.CUBiT.athttp://www.CUBiT.at
_______________________________________________ Toasters mailing list Toasters@teaparty.netjavascript:return http://www.teaparty.net/mailman/listinfo/toasters
I’ll have to agree as well, we’re recommending all of our customers go to either 8.1.3 or 8.2P3. We’ve had several people suffer from the dedupe fingerprinting issue in 8.1.2 and now there’s a serious data corruption bug (#723354) in 8.2 pre-P3.
Bug description:
o burt 723354http://burtweb-prd.eng.netapp.com/burt/burt-bin/start?burt-id=723354 “Illegal container vbn panic while loading a BLKR MOVED buf”; SW, dense, sev-2. A problem in all current 8.2 releases (8.2RC1, 8.2GA, 8.2P1, 8.2P2, and any D-patch derivatives) which may result in WAFL inconsistencies on systems running either 7-Mode or clustered Data ONTAP if they have dedupe-enabled volumes and there is repeated overwriting of 4KB blocks while a deduplication process is running against that volume. A is included in Data ONTAP 8.2P3 and existing customers using dedupe on 8.2 are advised to disable dedupe now to avoid further exposure, and then upgrade to 8.2P3 as soon as is operationally feasable (at which time dedupe can be re-enabled). It is also recommended that customers using dedupe on a pre-8.2 release of Data ONTAP that are considering upgrading to Data ONTAP 8.2 upgrade to 8.2P3 (or later) to avoid this issue.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser Sent: Friday, October 18, 2013 5:13 PM To: Fred Grieco; Mark Allen Cc: Peter-Paul Witta; toasters@teaparty.net Subject: AW: AW: Dedup killing performance (again)
Alright, I’m almost convinced to go for 8.1.3 now – will read through the changelogs tomorrow morning; right now, everything is fine again, fingerprint recovery has been finished a few minutes after my last mail regarding this issue and I’m now deleting some old snapshots to regain that space which should be fine for now.
I’ve disabled any future sis schedules for now until I know how I want to proceed with that situation.
Thanks for all your fast help as always ☺
Bye,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atmailto:ag@anexia.at Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Fred Grieco [mailto:fredgrieco@yahoo.com] Gesendet: Samstag, 19. Oktober 2013 02:11 An: Alexander Griesser; Mark Allen Cc: Peter-Paul Witta; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: AW: Dedup killing performance (again)
I have 8.1.3p1 on a 3140 and a 3270 (7 mode) with no dedupe performance issues.
Sent from Yahoo! Mail on Android
________________________________ From: Mark Allen <mmmjka@gmail.commailto:mmmjka@gmail.com>; To: Alexander Griesser <ag@anexia.atmailto:ag@anexia.at>; Cc: Peter-Paul Witta <paul.witta@cubit.atmailto:paul.witta@cubit.at>; <toasters@teaparty.netmailto:toasters@teaparty.net>; Subject: Re: AW: Dedup killing performance (again) Sent: Sat, Oct 19, 2013 12:00:06 AM
I would agree!! I upgraded to 8.1.3p2 on several arrays over the past month and there hasn't been any fall out.
-Mark On Oct 18, 2013 4:39 PM, "Alexander Griesser" <ag@anexia.atjavascript:return> wrote: Is it stable? I mean 8.1.3 is just in GA for now…
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Peter-Paul Witta [mailto:paul.witta@CUBiT.atjavascript:return] Gesendet: Samstag, 19. Oktober 2013 00:30 An: Alexander Griesser; Sebastian Goetze Cc: toasters@teaparty.netjavascript:return Betreff: Re: Dedup killing performance (again)
I'd recommend 813p2 immediately...
lg paul witta
Am 19.10.13 00:19, schrieb Alexander Griesser: Yah, just found it while googling I guess, must be bug id #657692 :-/
Anyhow, I’m now stuck with this:
sis status
Path State Status Progress /vol/CIFS_SHARES Enabled Active Fingerprint is being recovered from aggregate (6% complete)
It’s doing that since quite some time now, although it now actually jumped from 0% to 6%, so it at least is still alive.
This bug is first fixed in 8.1.2P4 as it seems, so I might need to upgrade ASAP; for now, I’ll check if it at least finishes this process now, my volume is filling up quickly, but there’s still some aggregate space left so I’ll just increase the volume size before it goes full. I’ll go above the magic 80% fill of an aggregate then, but better that then having the volume go full.
I’ll see how it goes and if it troubles me too much, I hope there is a way to abort sis in that state.
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Sebastian Goetze [mailto:spgoetze@gmail.comjavascript:return] Gesendet: Samstag, 19. Oktober 2013 00:16 An: Alexander Griesser Betreff: Re: Dedup killing performance (again)
Isn't that the Dedupe bug again, that was fixed in 8.1.3? (since like 8.1.2P3 or P4)...
There was a thread about it some time ago...
Anyway, I'm off to bed, just wanted to point you in the right direction.
BFN On 10/18/2013 11:52 PM, Alexander Griesser wrote: Hi there,
I am just trying to dedupe some volumes on one of our filers and the first volume (2.4TB data, CIFS share) deduplicates since about 4 hours now. Sysstat –x 1 shows:
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk OTHER FCP iSCSI FCP kB/s iSCSI kB/s in out read write read write age hit time ty util in out in out 58% 115 0 0 116 2239 3635 281593 32 0 0 0s 98% 0% - 100% 0 0 1 0 0 0 0 63% 162 0 0 169 1682 4149 310829 0 0 0 0s 99% 0% - 100% 7 0 0 0 0 0 0 50% 381 0 0 384 11793 11325 182100 0 0 0 0s 96% 0% - 100% 0 0 3 0 0 0 0 55% 412 0 0 416 6487 6040 223036 24 0 0 0s 97% 0% - 100% 0 0 4 0 0 37 0 51% 19 0 0 19 185 9 239388 0 0 0 2 98% 0% - 100% 0 0 0 0 0 0 0 51% 77 0 0 80 348 906 273880 8 0 0 2 99% 0% - 100% 0 0 3 0 0 0 0 60% 48 0 0 49 174 410 196179 271 0 0 2 98% 35% Tn 100% 0 0 1 0 0 0 0 45% 102 0 0 110 337 337 144387 21222 0 0 2 96% 100% :f 100% 8 0 0 0 0 0 0 71% 143 0 0 149 258 2814 109728 24268 0 0 2 85% 100% :f 86% 0 0 6 0 0 25 0 53% 64 0 0 65 203 1368 236620 14152 0 0 2 98% 100% :f 100% 0 0 1 0 0 0 0 49% 96 0 0 96 319 1317 185552 19748 0 0 2 96% 100% :f 100% 0 0 0 0 0 0 0 35% 109 0 0 117 381 1962 62613 18010 0 0 2 88% 100% :f 100% 5 0 3 0 0 0 0 55% 38 0 0 39 159 1168 239408 1980 0 0 2 97% 100% :f 100% 0 0 1 0 0 0 0 57% 177 0 0 179 308 6692 136568 520 0 0 2 92% 82% : 99% 0 0 2 0 0 37 0 39% 256 0 0 271 403 12647 117284 0 0 0 2 93% 0% - 97% 12 0 3 0 0 0 0 40% 232 0 0 473 808 7506 162368 32 0 0 2 96% 0% - 100% 240 0 1 0 0 0 0 38% 86 0 0 86 246 3413 160092 0 0 0 2 96% 0% - 100% 0 0 0 0 0 0 0 47% 113 0 0 117 314 1738 199516 0 0 0 2 96% 0% - 100% 1 0 3 0 0 0 0 59% 116 0 0 120 476 2751 288048 24 0 0 0s 99% 0% - 100% 0 0 4 0 0 24 0 45% 230 0 0 230 2879 6547 185787 120 0 0 0s 98% 0% - 100% 0 0 0 0 0 0 0
version
NetApp Release 8.1.2P2 7-Mode: Wed Feb 20 19:58:27 PST 2013
I’m aware that deduplication _SHOULD_ be a background process hugging up as much resources as it can but when host i/o is needed, it will be preferred over deduplication. There are no other tasks running at the same time (snapvault, etc.), and the virtual machinse on this system are _VERY_ slow – f.ex. logging in via RDP on one of the virtual machines now took > 1 minute for the login window to appear.
This is a FAS2240 with 24x600 SAS 10k and basically my question now is: Is there any way to tell a-sis to really run with lower priority or is this a bug in the OnTap version I’m running? I’ve used a-sis a lot in the past and never had performance issues like this when it was running…
I’m thankful for any ideas how to track down this issue or improve performance while a-sis is running.
Thanks,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
_______________________________________________
Toasters mailing list
Toasters@teaparty.netjavascript:return
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.netjavascript:return
http://www.teaparty.net/mailman/listinfo/toasters
--
kind regards, +43-664-4542287
Ing. Peter-Paul Witta +43-1-7189880-0
Chief Technology Officer www.CUBiT.athttp://www.CUBiT.at
_______________________________________________ Toasters mailing list Toasters@teaparty.netjavascript:return http://www.teaparty.net/mailman/listinfo/toasters
JFTR: Upgraded to 8.1.3P2 yesterday, everything good again – dedupes are no longer causing performance issues, so this was the solution.
Bye,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atmailto:ag@anexia.at Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Im Auftrag von Aaron Lewis Gesendet: Samstag, 19. Oktober 2013 02:22 An: toasters@teaparty.net Betreff: RE: AW: Dedup killing performance (again)
I’ll have to agree as well, we’re recommending all of our customers go to either 8.1.3 or 8.2P3. We’ve had several people suffer from the dedupe fingerprinting issue in 8.1.2 and now there’s a serious data corruption bug (#723354) in 8.2 pre-P3.
Bug description:
o burt 723354http://burtweb-prd.eng.netapp.com/burt/burt-bin/start?burt-id=723354 “Illegal container vbn panic while loading a BLKR MOVED buf”; SW, dense, sev-2. A problem in all current 8.2 releases (8.2RC1, 8.2GA, 8.2P1, 8.2P2, and any D-patch derivatives) which may result in WAFL inconsistencies on systems running either 7-Mode or clustered Data ONTAP if they have dedupe-enabled volumes and there is repeated overwriting of 4KB blocks while a deduplication process is running against that volume. A is included in Data ONTAP 8.2P3 and existing customers using dedupe on 8.2 are advised to disable dedupe now to avoid further exposure, and then upgrade to 8.2P3 as soon as is operationally feasable (at which time dedupe can be re-enabled). It is also recommended that customers using dedupe on a pre-8.2 release of Data ONTAP that are considering upgrading to Data ONTAP 8.2 upgrade to 8.2P3 (or later) to avoid this issue.
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser Sent: Friday, October 18, 2013 5:13 PM To: Fred Grieco; Mark Allen Cc: Peter-Paul Witta; toasters@teaparty.netmailto:toasters@teaparty.net Subject: AW: AW: Dedup killing performance (again)
Alright, I’m almost convinced to go for 8.1.3 now – will read through the changelogs tomorrow morning; right now, everything is fine again, fingerprint recovery has been finished a few minutes after my last mail regarding this issue and I’m now deleting some old snapshots to regain that space which should be fine for now.
I’ve disabled any future sis schedules for now until I know how I want to proceed with that situation.
Thanks for all your fast help as always ☺
Bye,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atmailto:ag@anexia.at Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Fred Grieco [mailto:fredgrieco@yahoo.com] Gesendet: Samstag, 19. Oktober 2013 02:11 An: Alexander Griesser; Mark Allen Cc: Peter-Paul Witta; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: AW: Dedup killing performance (again)
I have 8.1.3p1 on a 3140 and a 3270 (7 mode) with no dedupe performance issues.
Sent from Yahoo! Mail on Android
________________________________ From: Mark Allen <mmmjka@gmail.commailto:mmmjka@gmail.com>; To: Alexander Griesser <ag@anexia.atmailto:ag@anexia.at>; Cc: Peter-Paul Witta <paul.witta@cubit.atmailto:paul.witta@cubit.at>; <toasters@teaparty.netmailto:toasters@teaparty.net>; Subject: Re: AW: Dedup killing performance (again) Sent: Sat, Oct 19, 2013 12:00:06 AM
I would agree!! I upgraded to 8.1.3p2 on several arrays over the past month and there hasn't been any fall out.
-Mark On Oct 18, 2013 4:39 PM, "Alexander Griesser" <ag@anexia.atjavascript:return> wrote: Is it stable? I mean 8.1.3 is just in GA for now…
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Peter-Paul Witta [mailto:paul.witta@CUBiT.atjavascript:return] Gesendet: Samstag, 19. Oktober 2013 00:30 An: Alexander Griesser; Sebastian Goetze Cc: toasters@teaparty.netjavascript:return Betreff: Re: Dedup killing performance (again)
I'd recommend 813p2 immediately...
lg paul witta
Am 19.10.13 00:19, schrieb Alexander Griesser: Yah, just found it while googling I guess, must be bug id #657692 :-/
Anyhow, I’m now stuck with this:
sis status
Path State Status Progress /vol/CIFS_SHARES Enabled Active Fingerprint is being recovered from aggregate (6% complete)
It’s doing that since quite some time now, although it now actually jumped from 0% to 6%, so it at least is still alive.
This bug is first fixed in 8.1.2P4 as it seems, so I might need to upgrade ASAP; for now, I’ll check if it at least finishes this process now, my volume is filling up quickly, but there’s still some aggregate space left so I’ll just increase the volume size before it goes full. I’ll go above the magic 80% fill of an aggregate then, but better that then having the volume go full.
I’ll see how it goes and if it troubles me too much, I hope there is a way to abort sis in that state.
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Sebastian Goetze [mailto:spgoetze@gmail.comjavascript:return] Gesendet: Samstag, 19. Oktober 2013 00:16 An: Alexander Griesser Betreff: Re: Dedup killing performance (again)
Isn't that the Dedupe bug again, that was fixed in 8.1.3? (since like 8.1.2P3 or P4)...
There was a thread about it some time ago...
Anyway, I'm off to bed, just wanted to point you in the right direction.
BFN On 10/18/2013 11:52 PM, Alexander Griesser wrote: Hi there,
I am just trying to dedupe some volumes on one of our filers and the first volume (2.4TB data, CIFS share) deduplicates since about 4 hours now. Sysstat –x 1 shows:
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk OTHER FCP iSCSI FCP kB/s iSCSI kB/s in out read write read write age hit time ty util in out in out 58% 115 0 0 116 2239 3635 281593 32 0 0 0s 98% 0% - 100% 0 0 1 0 0 0 0 63% 162 0 0 169 1682 4149 310829 0 0 0 0s 99% 0% - 100% 7 0 0 0 0 0 0 50% 381 0 0 384 11793 11325 182100 0 0 0 0s 96% 0% - 100% 0 0 3 0 0 0 0 55% 412 0 0 416 6487 6040 223036 24 0 0 0s 97% 0% - 100% 0 0 4 0 0 37 0 51% 19 0 0 19 185 9 239388 0 0 0 2 98% 0% - 100% 0 0 0 0 0 0 0 51% 77 0 0 80 348 906 273880 8 0 0 2 99% 0% - 100% 0 0 3 0 0 0 0 60% 48 0 0 49 174 410 196179 271 0 0 2 98% 35% Tn 100% 0 0 1 0 0 0 0 45% 102 0 0 110 337 337 144387 21222 0 0 2 96% 100% :f 100% 8 0 0 0 0 0 0 71% 143 0 0 149 258 2814 109728 24268 0 0 2 85% 100% :f 86% 0 0 6 0 0 25 0 53% 64 0 0 65 203 1368 236620 14152 0 0 2 98% 100% :f 100% 0 0 1 0 0 0 0 49% 96 0 0 96 319 1317 185552 19748 0 0 2 96% 100% :f 100% 0 0 0 0 0 0 0 35% 109 0 0 117 381 1962 62613 18010 0 0 2 88% 100% :f 100% 5 0 3 0 0 0 0 55% 38 0 0 39 159 1168 239408 1980 0 0 2 97% 100% :f 100% 0 0 1 0 0 0 0 57% 177 0 0 179 308 6692 136568 520 0 0 2 92% 82% : 99% 0 0 2 0 0 37 0 39% 256 0 0 271 403 12647 117284 0 0 0 2 93% 0% - 97% 12 0 3 0 0 0 0 40% 232 0 0 473 808 7506 162368 32 0 0 2 96% 0% - 100% 240 0 1 0 0 0 0 38% 86 0 0 86 246 3413 160092 0 0 0 2 96% 0% - 100% 0 0 0 0 0 0 0 47% 113 0 0 117 314 1738 199516 0 0 0 2 96% 0% - 100% 1 0 3 0 0 0 0 59% 116 0 0 120 476 2751 288048 24 0 0 0s 99% 0% - 100% 0 0 4 0 0 24 0 45% 230 0 0 230 2879 6547 185787 120 0 0 0s 98% 0% - 100% 0 0 0 0 0 0 0
version
NetApp Release 8.1.2P2 7-Mode: Wed Feb 20 19:58:27 PST 2013
I’m aware that deduplication _SHOULD_ be a background process hugging up as much resources as it can but when host i/o is needed, it will be preferred over deduplication. There are no other tasks running at the same time (snapvault, etc.), and the virtual machinse on this system are _VERY_ slow – f.ex. logging in via RDP on one of the virtual machines now took > 1 minute for the login window to appear.
This is a FAS2240 with 24x600 SAS 10k and basically my question now is: Is there any way to tell a-sis to really run with lower priority or is this a bug in the OnTap version I’m running? I’ve used a-sis a lot in the past and never had performance issues like this when it was running…
I’m thankful for any ideas how to track down this issue or improve performance while a-sis is running.
Thanks,
Alexander Griesser System-Administrator
ANEXIA Internetdienstleistungs GmbH
Telefon: +43-5-0556-320 Telefax: +43-5-0556-500
E-Mail: ag@anexia.atjavascript:return Web: http://www.anexia.athttp://www.anexia.at/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
_______________________________________________
Toasters mailing list
Toasters@teaparty.netjavascript:return
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.netjavascript:return
http://www.teaparty.net/mailman/listinfo/toasters
--
kind regards, +43-664-4542287
Ing. Peter-Paul Witta +43-1-7189880-0
Chief Technology Officer www.CUBiT.athttp://www.CUBiT.at
_______________________________________________ Toasters mailing list Toasters@teaparty.netjavascript:return http://www.teaparty.net/mailman/listinfo/toasters
-
Aaron Lewis -
Alexander Griesser -
Fred Grieco -
Mark Allen