Performance Comparison Between WinSCP and WGET

Scenario:
Standard FTP server and path: ftp://ftp.lip6.fr/pub/doc/
Single thread batch/script
Mirroring all folders and files

First Mirror Stats (initial file copy):
WinSCP: 110 minutes - winscp_log_1.txt
WGET: 86 minutes - wget_log_1.txt

Second Mirror Stats (file compare with no copy):
WinSCP: 22 minutes - winscp_log_2.txt
WGET: 15 minutes - wget_log_2.txt

I was a bit surprised the difference was so large. Why is WinSCP underperforming so much?

Attached are the batch/script/log files.

So you see the difference when transferring a lot of small files like you did only? Or even when transferring one large file?

It is the small file transfers that have the largest difference.

With larger files, they are more comparable with WinSCP seeming to have an advantage. However with larger files, I am finding a very large swing in download times between runs which I suspect is due to the loading on the server so I am less confident in the direct performance comparison.

With the small file version, running it multiple times for the mirror check (i.e. file comparison without file download), the times are fairly consistent as well as across servers.

I was just trying to figure out if something else is going on during a file comparison that would explain the difference in times.

Can you post more verbose logs? Not /loglevel=-1 in WinSCP and equivalent in wget.

Here you go!

Thanks. Though those are not logs of files transfers.

martin wrote:

Thanks. Though those are not logs of files transfers.

Correct, they are the log files for a mirror of the FTP server, looking for new files after the initial file transfer. While the initial file transfer is also important, the mirror is probably more important because it is done fairly frequently.

I had originally performed tests for both the initial file transfer as well as the follow up mirroring and found that WinSCP was slower in both cases.

Also, in the original zip that I posted are the batch files for running both application that can be modified for your local file system so you can do your own testing if needed.

I seems that it takes wget about 1.1s to process one directory, while it takes WinSCP about 1.4s.
I do not know if wget logs all commands it sends to the server. I'd assume it does.
Then WinSCP does two additional commands per directory – PWD and TYPE A. These take about 0.25s to process, what corresponds to the difference.

Thanks for the update!