Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I'm exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague Chris_Hague@ajg.com wrote:
Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe?
Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance,
Chris.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
If you use robocopy I used the following switches:
/MIR /e /copyall /r:0 /w:0
/mir is a dangerous one; make sure you understand it.. it might make your day go bad. Copyall copies perms PLUS ownership. Worked like a charm (at least for me back in the day).
From: toasters-bounces@teaparty.net on behalf of Basil basilberntsen@gmail.com Date: Tuesday, April 11, 2017 at 1:44 PM To: Chris Hague Chris_Hague@ajg.com, "toasters@teaparty.net" toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
robocopy /E /COPYALL /MT:8 /R:1 /W:1 /MIR /LOG:c:\path /ZB X:\source y:\dest
/E : Copy Subfolders, including Empty Subfolders. /COPYALL : Copy ALL file info (equivalent to /COPY:DATSOU) /MT:8 -> multi-threaded….how may cpu threads to use? 1-16 /R:1 -> how many retries on failure /W:1 -> how long to wait between failures */J : Copy using unbuffered I/O (recommended for large files).* /ZB : Use restartable mode; if access denied use Backup mode. /LOG:c:\path -> where to log info to. Much faster than putting on screen. dump to file and check file when done.
--> if a subsequent run is done, use /LOG+:c:\path to append to original file.
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
On Tue, Apr 11, 2017 at 4:51 PM, Klise, Steve Steve.Klise@wwt.com wrote:
If you use robocopy I used the following switches:
/MIR /e /copyall /r:0 /w:0
/mir is a dangerous one; make sure you understand it.. it might make your day go bad. Copyall copies perms PLUS ownership. Worked like a charm (at least for me back in the day).
*From: *toasters-bounces@teaparty.net on behalf of Basil < basilberntsen@gmail.com> *Date: *Tuesday, April 11, 2017 at 1:44 PM *To: *Chris Hague Chris_Hague@ajg.com, "toasters@teaparty.net" < toasters@teaparty.net> *Subject: *Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague Chris_Hague@ajg.com wrote:
Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe?
Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance,
Chris.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Hey Toasters, This tidbit of important info has recently come to my attention:
Please be aware: There is an incompatibility between certain versions of robocopy and certain versions of clustered Data ONTAP that could result in files larger than 512MiB not being copied correctly. The incompatibility exists between versions of ONTAP starting with 8.3.1, and with the version of robocopy distributed with Windows 7 and Windows Server 2008R2 (internal version 5.1.10.1027). The robocopy distributed with later versions of Windows does not exhibit this problem.
So, if you are planning on using Robocopy, Use a windows version newer than windows 7 / server 2008R2.
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
443-228-TMAC (*Google Voice*) 214-279-3926 (*eFAX*)
[image: NetApp - In partnership with Alpine Testing Solutions][image: NetApp Certified Data Administrator, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-05&i=35&ci=NETAPP00041276[image: NetApp Certified Implementation Engineer - SAN Specialist, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-08&i=36&ci=NETAPP00041276[image: NetApp Certified Storage Installation Engineer, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-13&i=38&ci=NETAPP00041276[image: NetApp Certified Implementation Engineer - Data Protection Specialist] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-15&i=11&ci=NETAPP00041276
NetApp Candidate ID: NETAPP00041276 FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH
RHCE6 110-107-141 https://www.redhat.com/wapps/training/certification/verify.html?certNumber=110-107-141&isSearch=False&verify=Verify
On Tue, Apr 11, 2017 at 5:10 PM, tmac tmacmd@gmail.com wrote:
robocopy /E /COPYALL /MT:8 /R:1 /W:1 /MIR /LOG:c:\path /ZB X:\source y:\dest
/E : Copy Subfolders, including Empty Subfolders. /COPYALL : Copy ALL file info (equivalent to /COPY:DATSOU) /MT:8 -> multi-threaded….how may cpu threads to use? 1-16 /R:1 -> how many retries on failure /W:1 -> how long to wait between failures */J : Copy using unbuffered I/O (recommended for large files).* /ZB : Use restartable mode; if access denied use Backup mode. /LOG:c:\path -> where to log info to. Much faster than putting on screen. dump to file and check file when done.
--> if a subsequent run is done, use /LOG+:c:\path to append to original file.
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
On Tue, Apr 11, 2017 at 4:51 PM, Klise, Steve Steve.Klise@wwt.com wrote:
If you use robocopy I used the following switches:
/MIR /e /copyall /r:0 /w:0
/mir is a dangerous one; make sure you understand it.. it might make your day go bad. Copyall copies perms PLUS ownership. Worked like a charm (at least for me back in the day).
*From: *toasters-bounces@teaparty.net on behalf of Basil < basilberntsen@gmail.com> *Date: *Tuesday, April 11, 2017 at 1:44 PM *To: *Chris Hague Chris_Hague@ajg.com, "toasters@teaparty.net" < toasters@teaparty.net> *Subject: *Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague Chris_Hague@ajg.com wrote:
Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe?
Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance,
Chris.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell –node a cd /folder rm –rf ☺
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares. Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far. The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Im Auftrag von Chris Hague Gesendet: Mittwoch, 12. April 2017 17:52 An: toasters@teaparty.net Betreff: RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell –node a cd /folder rm –rf ☺
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Something I did last year...
Download XCP from netapp support. Run on a LINUX machine with a high speed connection. Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.
Then go back over with something like robocopy to apply permissions? If I recall, robocopy will not copy the file, just the security features of the file.
Just a thought.
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <AGriesser@anexia-it.com
wrote:
In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.
Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.
The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] *Im Auftrag von *Chris Hague *Gesendet:* Mittwoch, 12. April 2017 17:52 *An:* toasters@teaparty.net *Betreff:* RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share
subinacl.exe /file %wrkfile% /setowner=%username%
subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username%
ICACLS %wrkfile% /grant %username%:F /T
del /f /s /q %wrkfile%
rmdir /q /s %wrkfile%
FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way??
systemshell –node a
cd /folder
rm –rf
J
*From:* toasters-bounces@teaparty.net [mailto:toasters-bounces@ teaparty.net toasters-bounces@teaparty.net] *On Behalf Of *Chris Hague *Sent:* 12 April 2017 09:20 *To:* Basil; toasters@teaparty.net *Subject:* RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/ fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards,
Chris.
*From:* Basil [mailto:basilberntsen@gmail.com basilberntsen@gmail.com] *Sent:* 11 April 2017 21:45 *To:* Chris Hague; toasters@teaparty.net *Subject:* Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague Chris_Hague@ajg.com wrote:
Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe?
Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance,
Chris.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.com] Gesendet: Mittwoch, 12. April 2017 19:23 An: Alexander Griesser AGriesser@anexia-it.com Cc: Chris Hague Chris_Hague@ajg.com; toasters@teaparty.net Betreff: Re: 7MTT CIFS Question
Something I did last year...
Download XCP from netapp support. Run on a LINUX machine with a high speed connection. Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.
Then go back over with something like robocopy to apply permissions? If I recall, robocopy will not copy the file, just the security features of the file.
Just a thought.
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares. Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far. The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] Im Auftrag von Chris Hague Gesendet: Mittwoch, 12. April 2017 17:52 An: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell –node a cd /folder rm –rf ☺
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
XCP needs a license, but it’s free.
The license is for tracking and statistic purposes only.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser Sent: Wednesday, April 12, 2017 1:25 PM To: NGC-tmacmd-gmail.com tmacmd@gmail.com Cc: toasters@teaparty.net Subject: AW: 7MTT CIFS Question
Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.com] Gesendet: Mittwoch, 12. April 2017 19:23 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: 7MTT CIFS Question
Something I did last year...
Download XCP from netapp support. Run on a LINUX machine with a high speed connection. Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.
Then go back over with something like robocopy to apply permissions? If I recall, robocopy will not copy the file, just the security features of the file.
Just a thought.
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares. Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far. The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] Im Auftrag von Chris Hague Gesendet: Mittwoch, 12. April 2017 17:52 An: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell –node a cd /folder rm –rf ☺
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Alright, great – will ask my sales rep to send me a bunch of those then ☺
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Parisi, Justin [mailto:Justin.Parisi@netapp.com] Gesendet: Mittwoch, 12. April 2017 19:26 An: Alexander Griesser AGriesser@anexia-it.com; NGC-tmacmd-gmail.com tmacmd@gmail.com Cc: toasters@teaparty.net Betreff: RE: 7MTT CIFS Question
XCP needs a license, but it’s free.
The license is for tracking and statistic purposes only.
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser Sent: Wednesday, April 12, 2017 1:25 PM To: NGC-tmacmd-gmail.com <tmacmd@gmail.commailto:tmacmd@gmail.com> Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: AW: 7MTT CIFS Question
Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.com] Gesendet: Mittwoch, 12. April 2017 19:23 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: 7MTT CIFS Question
Something I did last year...
Download XCP from netapp support. Run on a LINUX machine with a high speed connection. Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.
Then go back over with something like robocopy to apply permissions? If I recall, robocopy will not copy the file, just the security features of the file.
Just a thought.
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares. Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far. The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] Im Auftrag von Chris Hague Gesendet: Mittwoch, 12. April 2017 17:52 An: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell –node a cd /folder rm –rf ☺
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Just ask. The license is free. If it runs out, ask for another. They should just give it to you They just do that to track who and how the software is being used.
Get Outlook for iOShttps://aka.ms/o0ukef ________________________________ From: Alexander Griesser AGriesser@anexia-it.com Sent: Wednesday, April 12, 2017 1:24:55 PM To: tmac Cc: Chris Hague; toasters@teaparty.net Subject: AW: 7MTT CIFS Question
Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.com] Gesendet: Mittwoch, 12. April 2017 19:23 An: Alexander Griesser AGriesser@anexia-it.com Cc: Chris Hague Chris_Hague@ajg.com; toasters@teaparty.net Betreff: Re: 7MTT CIFS Question
Something I did last year...
Download XCP from netapp support. Run on a LINUX machine with a high speed connection. Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.
Then go back over with something like robocopy to apply permissions? If I recall, robocopy will not copy the file, just the security features of the file.
Just a thought.
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares. Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far. The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] Im Auftrag von Chris Hague Gesendet: Mittwoch, 12. April 2017 17:52 An: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell –node a cd /folder rm –rf :)
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
You don't even need to ask. Go to xcp.netapp.com and request one there. Then download and enjoy.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Tim McCarthy Sent: Wednesday, April 12, 2017 1:27 PM To: Alexander Griesser agriesser@anexia-it.com Cc: toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Just ask. The license is free. If it runs out, ask for another. They should just give it to you They just do that to track who and how the software is being used.
Get Outlook for iOShttps://aka.ms/o0ukef ________________________________ From: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Sent: Wednesday, April 12, 2017 1:24:55 PM To: tmac Cc: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: AW: 7MTT CIFS Question
Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.com] Gesendet: Mittwoch, 12. April 2017 19:23 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: 7MTT CIFS Question
Something I did last year...
Download XCP from netapp support. Run on a LINUX machine with a high speed connection. Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.
Then go back over with something like robocopy to apply permissions? If I recall, robocopy will not copy the file, just the security features of the file.
Just a thought.
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares. Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far. The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process - but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] Im Auftrag von Chris Hague Gesendet: Mittwoch, 12. April 2017 17:52 An: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: 7MTT CIFS Question
Ok, so fastcopy isn't really much faster than windows delete. However it does delete the "path too long" files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I'm still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell -node a cd /folder rm -rf :)
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files... I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I'm exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
At least on the version I use, xcp is nfs only. The secret sauce is that they don't use the OS for nfs, they use python. You can do anything they do yourself with python in a vm.
Be careful of trying to use nfs on ntfs security style files- test first. If you want to use nfs to delete, you might be able to change the security style of the data to unix, which will drastically simplify your task. Once everything is unix security, being root on a server which isn't being rootsquashed should allow you to delete everything.
On Wed, Apr 12, 2017 at 1:34 PM Parisi, Justin Justin.Parisi@netapp.com wrote:
You don’t even need to ask. Go to xcp.netapp.com and request one there. Then download and enjoy.
*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *Tim McCarthy *Sent:* Wednesday, April 12, 2017 1:27 PM *To:* Alexander Griesser agriesser@anexia-it.com *Cc:* toasters@teaparty.net
*Subject:* Re: 7MTT CIFS Question
Just ask. The license is free. If it runs out, ask for another. They should just give it to you
They just do that to track who and how the software is being used.
Get Outlook for iOS https://aka.ms/o0ukef
*From:* Alexander Griesser AGriesser@anexia-it.com *Sent:* Wednesday, April 12, 2017 1:24:55 PM *To:* tmac *Cc:* Chris Hague; toasters@teaparty.net *Subject:* AW: 7MTT CIFS Question
Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* tmac [mailto:tmacmd@gmail.com tmacmd@gmail.com] *Gesendet:* Mittwoch, 12. April 2017 19:23 *An:* Alexander Griesser AGriesser@anexia-it.com *Cc:* Chris Hague Chris_Hague@ajg.com; toasters@teaparty.net *Betreff:* Re: 7MTT CIFS Question
Something I did last year...
Download XCP from netapp support. Run on a LINUX machine with a high speed connection.
Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.
Then go back over with something like robocopy to apply permissions?
If I recall, robocopy will not copy the file, just the security features of the file.
Just a thought.
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at **TMACsRack https://tmacsrack.wordpress.com/*
On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser < AGriesser@anexia-it.com> wrote:
In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.
Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.
The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] *Im Auftrag von *Chris Hague *Gesendet:* Mittwoch, 12. April 2017 17:52 *An:* toasters@teaparty.net *Betreff:* RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share
subinacl.exe /file %wrkfile% /setowner=%username%
subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username%
ICACLS %wrkfile% /grant %username%:F /T
del /f /s /q %wrkfile%
rmdir /q /s %wrkfile%
FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way??
systemshell –node a
cd /folder
rm –rf
J
*From:* toasters-bounces@teaparty.net [ mailto:toasters-bounces@teaparty.net toasters-bounces@teaparty.net] *On Behalf Of *Chris Hague *Sent:* 12 April 2017 09:20 *To:* Basil; toasters@teaparty.net *Subject:* RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards,
Chris.
*From:* Basil [mailto:basilberntsen@gmail.com basilberntsen@gmail.com] *Sent:* 11 April 2017 21:45 *To:* Chris Hague; toasters@teaparty.net *Subject:* Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague Chris_Hague@ajg.com wrote:
Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe?
Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance,
Chris.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
On 2017-04-12 19:24, Alexander Griesser wrote:
Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?
As many have already pointed out, is quite easy to just get an XCP license from a NetApp Web-portal. So, yes (I think) :-) XCP is a very good and impressive tool, its author certainly knows what he's doing (Peter Schay)
/M
Just a quick update on this one for anyone interested;
Having left the script to take ownership and change permissions running in windows for over a week on one folder containing 22m files, it had not even gotten past the first 1 of 3160 root folders!!!!!!
So, with the aid of an ONTAP 9 simulator, we were able to rm –r –f from the systemshell.
You have to unlock the diag user, then “sudo rm –r –f /clust/svm/volume” but it works and works without requiring ownership or permission to the files. It’s fast too. (done nearly 140 root folders in an hour)
This isn’t ideal having to use the systemshell, but is the only way we have found to perform this task in a timely, reliable manor.
Kind Regards, Chris.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 16:52 To: toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.
Now I have permissions issues!
So I have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.
set wrkfile=\cifs_server\root_share\dead_share subinacl.exe /file %wrkfile% /setowner=%username% subinacl.exe /subdirectories %wrkfile%*.* /setowner=%username% ICACLS %wrkfile% /grant %username%:F /T del /f /s /q %wrkfile% rmdir /q /s %wrkfile% FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%
But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.
Surely there has got to be an easier way?? systemshell –node a cd /folder rm –rf ☺
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Hague Sent: 12 April 2017 09:20 To: Basil; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: 7MTT CIFS Question
Thanks Basil,
I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.
Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.
I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.
Kind Regards, Chris.
From: Basil [mailto:basilberntsen@gmail.com] Sent: 11 April 2017 21:45 To: Chris Hague; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: 7MTT CIFS Question
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone.
On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <Chris_Hague@ajg.commailto:Chris_Hague@ajg.com> wrote: Hi all,
I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.
My initial thoughts were to use robocopy, but there are potentially permissions issues.
Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.
Firstly, am I missing some simpler way of doing this? systemshell maybe? Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.
Many Thanks in advance, Chris.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Alexander Griesser -
Basil -
Chris Hague -
Klise, Steve -
Michael Bergman -
Parisi, Justin -
Tim McCarthy -
tmac