[Solved] Filezilla slower on Linux (compared to Windows)

[faster5on5windows]

Windows 8.1/10 64bit Filezilla 3.17.01
-Filezilla ftp and sftp download at 12 megabytes (96 megabits) per second no matter how many files I download. If I download 1 file then it downloads at 12 megabytes. 10 files download at about 1.2 megabytes each. Filezilla version is 3.17.01 but I also had the same speed with previous versions last year.

Ubuntu 16.04 64 bit Filezilla 3.15
-Filezilla sftp and regular ftp are limited to 800KB/s per file. If I download 10 files at once then I can get 8MB/s total. If I download only 1 file then download speed is stuck at 800KB/s (100% of the time no matter what seedbox I use).
-http/https downloads from seedboxes through Firefox and Chromium are just as fast as Windows (12MB/s)
-disabling UFW (firewall) does not help

Similar posts of Filezilla being slower on Linux
viewtopic.php?t=32022
OP kind of improves Linux speeds with motherboard setting that I do not have https://bbs.archlinux.org/viewtopic.php?id=198428

[botg]

Please update to the most recent version of FileZilla on your Linux systems. There have been significant SFTP improvements in recent versions.

Also check your sysctl knobs for TCP receive memory limits, they are probably all set too low. Iirc you need to increase net.core.rmem_max and net.ipv4.tcp_rmem

[faster5on5windows]

Please update to the most recent version of FileZilla on your Linux systems. There have been significant SFTP improvements in recent versions.

I get the same 800KB limit using plain ftp as well. I'm also sure that my SFTP speeds were not limited on Windows version 3.11. I will try installing the newest version when I get home though.

Also check your sysctl knobs for TCP receive memory limits, they are probably all set too low. Iirc you need to increase net.core.rmem_max and net.ipv4.tcp_rmem

I am very sorry but maybe my English is not enough to understand. Are these Linux settings? How would I change them?

[botg]

As root, execute this command to show the current limits:

Code: Select all

sysctl -a | grep net.*rmem

To increase them use sysctl -w, e.g. like this:

Code: Select all

sysctl -w net.ipv4.tcp_rmem=''40960 873800 62914560'

[faster5on5windows]

As root, execute this command to show the current limits:

Code: Select all

sysctl -a | grep net.*rmem

To increase them use sysctl -w, e.g. like this:

Code: Select all

sysctl -w net.ipv4.tcp_rmem=''40960 873800 62914560'

Thank you so much!
The issue was solved with:
sysctl -w net.ipv4.tcp_rmem='40960 873800 62914560'
sysctl -w net.core.rmem_max=8388608

Question 1. Is that the best number for net.core.rmem_max? The default was 212992.

Question 2. It turns out lftp DOES use segmented downloading by default. However without segmented downloading [pget -n 1 filename] and without the sysctl mod, lftp is still faster than filezilla (1.8MB/s per file vs 800KB/s. Any idea why this is?

After the mod speeds changed to-
-filezilla max 4MB per file, speed average 3MB per file. (Increase from 800KB per file). No difference between ftp and sftp.
-lftp 1.8MB/s (not segmented). (No increase)

Question 3. Why does Filezilla need this mod when no other program or type of download needs it or is effected by it? (except for lftp but this mod doesn't seem to effect lftp's speed).

Question 4. Do these mods have any other effects (increased chance of file corruption etc)

Question 5. If there are no negative effects to these settings then why are they not default?

[botg]

Background: A single connections speed is limited by the minimum of the connection's advertised TCP window and the bandwidth-delay product of the path through the network.

FileZilla requests a large receive window so that connections can be fast even on long fat networks with a huge bandwidth-delay product, e.g. fast transatlantic connections.

Question 1. Is that the best number for net.core.rmem_max? The default was 212992.

The best would probably be a number that's slightly higher than the largest bandwitdth-delay product you would ever encounter. Rule-of-thumb I use: Internet connection's maximum receive speed in bytes/s divided by 3. Unless you want to exchange files with the secret base on the far side of the moon, larger values have no benefit.

Question 2. [...] However without segmented downloading [pget -n 1 filename] and without the sysctl mod, lftp is still faster than filezilla (1.8MB/s per file vs 800KB/s. Any idea why this is?
Question 3. Why does Filezilla need this mod when no other program or type of download needs it or is effected by it? (except for lftp but this mod doesn't seem to effect lftp's speed).

There is a bug in the Linux kernel. For some reasons programs requesting a large TCP receive buffer actually get a smaller TCP window scale factor than programs not specifying a receive buffer at all.

(Technically, specifying the receive buffer shouldn't even be needed, but unfortunately the sockets API doesn't have any method of independently specifying the initial advertised receive window and receive window scale)

Question 4. Do these mods have any other effects (increased chance of file corruption etc)
Question 5. If there are no negative effects to these settings then why are they not default?

The main drawback is increased memory consumption. If more memory is used by each connection, a fewer total number of connections can be established. The defaults are lowest-common-denominator values and as far as I know haven't been updated in years, so they no longer adequately represent the capabilities of modern hardware and internet connections.

[faster5on5windows]

Thank you so much for such detailed answers! If you had not taught me this solution I was planning to buy a new motherboard
(´-﹏-`；)

[kotton]

OMG thank you so much for this! It's been driving me crazy and almost made me go back to windows because the fastest speed I could get a single file was 600kb/s. with the changes from this I can now get single files coming in at 2MB/s and 5 files at around 1MB/sec each.

thanks faster5on5windows for asking the question with so much research and botg you're a fucking legend!

Just one question, it doesn't seem to save the settings after a reboot. How do you make it permanent?

[maglar]

OMG thank you so much for this! It's been driving me crazy and almost made me go back to windows because the fastest speed I could get a single file was 600kb/s. with the changes from this I can now get single files coming in at 2MB/s and 5 files at around 1MB/sec each.

thanks faster5on5windows for asking the question with so much research and botg you're a fucking legend!

Just one question, it doesn't seem to save the settings after a reboot. How do you make it permanent?

I know I'm quite late to the party, but I also have been having this issue and found the solution to it thanks to a combination of the Ubuntu forums and the FileZilla forums. You can easily fix this problem by adding the commands to the end of your sysctl file.

In terminal:

Code: Select all

sudo nano /etc/sysctl.conf

Add the following lines to the end of that file:

Code: Select all

net.ipv4.tcp_rmem=40960 873800 62914560
net.core.rmem_max=25000000

I am running on Ubuntu 14.04 LTS and this worked for me. I don't see why it wouldn't work on Ubuntu 16.04 as well, but I have not tried myself. Kudos to jim_deadlock over at Ubuntu forums for posting the solution (it's the very last post).