Page 1 of 1

Concurrent Connections

Posted: 2019-12-09 21:50
by kedarj
I am going to fetch thousand of files in a day from 1400 directories ( each minute 1 directory) and all have images.

I tried the Site manager --> Transfer setting -> uncheck the box ( so not to limit). When I put the folder with 188 files ( size 2 MB total) . it take 45 seconds. That is really slow for 2 MB. ( I teted with 1 big file of 10 MB, it took 3 seconds) so its not the server / bandwidth / internet issue.

I am thinking it is not using the maximum connections. How to force File zilla to use all 10 connections at same time to it can bring in parallel. And How can I let it do background sync?

Re: Concurrent Connections

Posted: 2019-12-10 05:50
by boco
https://en.wikipedia.org/wiki/Protocol_ ... _overhead)

FTP is very bad with transferring many small files, due to the protocol overhead involved. Personally, I find 4 parallel transfers the best option, everything above gives rapidly diminishing returns. That's why it's capped at a max of 10, above 10, overhead would have a big hit on performance.
Thus, depending on situation, lowering parallel transfers might actually make it go faster.
How to force File zilla to use all 10 connections at same time to it can bring in parallel.
Not possible, it goes as fast as it can. But, as written above, managing 10 transfer connections to the server can hurt performance, especially with not high-end hardware (which offices rarely have).
And How can I let it do background sync?
Automatic transfers are not possible with FileZilla.