Need input if WINSCP would be the best option

I have a folder (with many sub-folders) & in some of those sub-folders there are hundreds of thousand, if not millions of small files (anywhere from 5kb-20kb hundreds of thousand document files in 1 folder, & millions of images ranging in size from 1mb-10mb in another folders which are also in sub folders dated per day)

I am trying to setup a way to sync the folders (1 way from source to target) & automate this to run every night to capture all the new files added to the source & then copy them to the traget.

I have initially done a baseline copy to a external nas, & shipped it to the location of the target machine. So right now the target machine has about 90% of the data, however it has been 2 months since this inital copy & the folders have grown approx. 40gb since then

the 2 sites are connected with a VPN.

My initial idea was to use something like Live Vault, however, it does not seem necessary to use it since there is a VPN connecting the 2 locations.

for smaller types of these migrations i use Richcopy & manually run it, or setup a schedule to do so, & then the night of the live cut over/migration i preform a initial sync of the 2 locations (1 way) & then restore their DB's

however, this client is 1 that plans on testing for a few months before going live on the new system & quite frankly i do not have the time to connect all the time & initiate the process.

any information would be greatly appreciated.



(also yes i know that hundred's of thousands of document files sitting in 1 directory is not the best thing on windows, but that is how their application does it.)

enohand wrote:

I have a folder (with many sub-folders) & in some of those sub-folders there are hundreds of thousand, if not millions of small files (anywhere from 5kb-20kb hundreds of thousand document files in 1 folder, & millions of images ranging in size from 1mb-10mb in another folders which are also in sub folders dated per day)

I am trying to setup a way to sync the folders (1 way from source to target) & automate this to run every night to capture all the new files added to the source & then copy them to the traget.

I have initially done a baseline copy to a external nas, & shipped it to the location of the target machine. So right now the target machine has about 90% of the data, however it has been 2 months since this inital copy & the folders have grown approx. 40gb since then

the 2 sites are connected with a VPN.

My initial idea was to use something like Live Vault, however, it does not seem necessary to use it since there is a VPN connecting the 2 locations.

for smaller types of these migrations i use Richcopy & manually run it, or setup a schedule to do so, & then the night of the live cut over/migration i preform a initial sync of the 2 locations (1 way) & then restore their DB's

however, this client is 1 that plans on testing for a few months before going live on the new system & quite frankly i do not have the time to connect all the time & initiate the process.

any information would be greatly appreciated.



(also yes i know that hundred's of thousands of document files sitting in 1 directory is not the best thing on windows, but that is how their application does it.)

i dont typically like to bump threads but i was hoping for a comment by now.


i wonder if using like Rsync, or Robocopy would be a better solution.


I tried reading through the manuals & stuff, but haven't been able to find out the info i need.


also i dont think i mentioned, that i can hit a UNC path to the target system, so i have no need for FTP, or any other protocol (unless it would be faster then the UNC path)