File copy sync issue - ideas ?

Hi

My issue does not fit neatly into any categort so i think this is the best fit.

>> The problem is how to provide the robocopy /MIR type functionality over multiple inputs..

Let me explain.

We have a new file server and i have a BAT file which came from Powershell to do the robocopy.

Source       Destination
\abc\bugs  \new\bugs
\abc\def\bugs \new\bugs
\hij\bnm\bugs \new\bugs

Due to the large file count and gigs. the robocopy was an all copy (not /MIR).
It takes days to run. MIR deletes files in the destination if they are not in the source.

At the change over point, i will run the script again and pick up the diffs. During the next week i am encouraging users to delete old content.
Ideally want i want to do is a combined /MIR functionality across multiple source locations to the same destination location.

>> So i am after tips / hints on how to do this ?

One way which should work, is to reset all the archive bits in " \new\bugs", do the copy setting the archive bit, then come through later and delete the files with no archive bit set.

This will because none of the files now exist in the multiple source destinations.

Thanks

July 12th, 2013 5:38am

Before thinking about a solution you need to state your problem in clear, unambiguous terms, e.g. like so:

  • I have a number of source folders, each containing a large number of bug report files.
  • I have a single destination folder.
  • Each day I wish to do this:
    1. Replace the bug reports in the destination folder with the bug reports from the source folders, if they are more recent.
    2. Delete those source folder bug reports that do not exist in the destination folder.
    3. Identify those destination folder bug reports that were added or modified in the current run.

If the above process is correct then you need to think hard about Step 1. Consider this case:

- d:\Source1\Report1.txt   10 July 2013
- d:\Source2\Report1.txt     9 July 2013
- e:\Target\Report1.txt        8 July 2013

When you run the sync process then the target file (8 July) will be replaced with the file from d:\Source1 (10 July). The file from Source2 will never make it to the target folder.

Free Windows Admin Tool Kit Click here and download it now
July 12th, 2013 6:19am

Don't use /MIR.  Use /COPY and select only copy attribute.
July 12th, 2013 8:56am

Thanks for your feedback, i thought it was clear when i wrote this but obviously i have not explained it clearly enough.

BIG COPY

Multiple copies have been done using /COPYALL, in some cases up to 5 separate disk areas have been copied to the one destination area.

OK so far all OK,

Meanwhile some tidy ups and deletes to be done in source areas.

FINAL RE-SYNC time

In the BIG COPY script is re-run, then any deleted content in the source will remain in the destination.
/MIR can only be used when there is one source / destimation entry, it is no good when multiple sources exist to the same destination.

>> So i was trying to explore what tricks i could use for the RE-SYNC (which only takes hours, not days) that resulted in deleted files in source being deleted from the destination.

regards


Free Windows Admin Tool Kit Click here and download it now
July 13th, 2013 3:44am

You have to write a script that compares every file in the target with all sources.  If the file does not exist in any source then delete it.

Start by generating a file with all source files by name\path. 

Next search target and lookup file in source.  If it cannot be found output name to a file of candidates for deletion.

This will be very fast as only one pass on all sources and destination will be made.  Since you will be generating a file of names to delete it can be run safely until you are sure it is what you want.

July 13th, 2013 6:13am

thanks jrv

Free Windows Admin Tool Kit Click here and download it now
July 13th, 2013 7:28pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics