Page 1 of 1
Filezilla stalls during upload
Posted: 2019-03-29 11:55
by hstoellinger
Hello,
Since about 2 weeks I experience pauses/stalls when trying to upload large numbers of files (eg- the Drupal core and Drupal vendor directoires and content). These contain mainly a lot of small files, but some 50 to 60 MB altogether. Weird enough, an even much larger transfer of something like 350 MB, but with mostly large files (eg. video or image files) transfers o.k, even though there also seems to be a pause before transferring the last couple of files completely. I have tried active and passive modes, have specified the sending of TCP-"keep-alive" messages, limited upload speeds, increased the number of parallel transfers, etc. etc.
The stall generally seems to occur when "receiving the content of a directory" (e.g. 12:43:26 Status: Empfange Verzeichnisinhalt für "/www_8613/core/lib/Drupal/Component/Datetime"...).
The transfer may then stop for minutes, with the server finally cancelling the connection after 5 minutes. It then reconnects again, but after a couple of transfers the same stall happens again...
Both client and server are Linux (Debian) systems. Max. upload speed is 10 Mbits/sec.
Has anybody got an idea what might cause this annoying behaviour?
Thanks in advance and greetings from Salzburg, Austria
H. Stoellinger
Re: Filezilla stalls during upload
Posted: 2019-03-29 12:06
by botg
Which version of FileZilla are you using?
What server software product and version are you connecting to?
Re: Filezilla stalls during upload
Posted: 2019-03-29 12:49
by hstoellinger
Hello,
Thanks for the quick reply!
I use the FileZilla-Client both from Debian 9 at version 3.24.0 and from Kubuntu (Ubuntu 18.10) at version 3.33.0 (latest packages distributed under both these distros). I connect to a Debian 8 system with proFTP version 1.3.7.
Regards
H. Stoellinger
Re: Filezilla stalls during upload
Posted: 2019-03-29 15:00
by botg
Both heavily outdated. As you're using Debian, please try the binaries for Debian from
https://filezilla-project.org/download. ... form=linux
Re: Filezilla stalls during upload
Posted: 2019-03-29 17:50
by hstoellinger
Hello,
Thanks - when I try the downloading site you suggest, I get a message which indicates that Google has found malware on the Filezilla site...
I suppose that this is a "false positive"! Correct?
Re: Filezilla stalls during upload
Posted: 2019-03-29 19:25
by hstoellinger
Hello again,
There aren't any NEWER versions on the sw-repositories of either Ubuntu or Debian. On the other hand the documentation on compiling from source is incomplete. It mentions ".configure" - which doesn't exist, it also prereqs some stuff - which is not found when trying to apt install it (havent't yet tried to find the packages for install with dpkg!). What a mess! Is there a chance to point me to a newer executable or even send it to me? (
hc.stoellinger@aon.at).
Thanks an awful lot for trying to be so helpful.
Good evening
H. S.
Re: Filezilla stalls during upload
Posted: 2019-03-29 20:37
by hstoellinger
Hello again,
I have now got 3.41.2 installed on my Kubuntu (Ubuntu 18.10) system -- same behavior!
Filezilla-client sends some "keep-alive" messages to the server after receiving the info about the contents of a directory. Then, after 300 secs the server closes the connection. I know that - according to specifications - the server should keep the idle TCP-control-message connection upright for much longer.
On the other hand, the REAL issue seems to be: Why does the transmission stall periodically in the first place? Suspecting some disk-hardware problem I transferred the data to be transmitted to another system - no difference!
Regards
H. S.
21:30:31 Status: Starte Upload von /mnt/hrm/www/core/modules/migrate_drupal_ui/src/Form/IdConflictForm.php
21:30:31 Status: Empfange Verzeichnisinhalt für "/www_8613/core/modules/migrate_drupal_ui/src/Form"... (= receiving directory content...)
21:30:40 Status: Sende Verbindungserhaltungs-Befehl (sending "keep.alive-messages")
21:31:10 Status: Sende Verbindungserhaltungs-Befehl
21:31:40 Status: Sende Verbindungserhaltungs-Befehl
21:32:10 Status: Sende Verbindungserhaltungs-Befehl
21:32:40 Status: Sende Verbindungserhaltungs-Befehl
21:33:10 Status: Sende Verbindungserhaltungs-Befehl
21:33:40 Status: Sende Verbindungserhaltungs-Befehl
..
21:34:10 Status: Sende Verbindungserhaltungs-Befehl
21:34:11 Befehl: NOOP
21:34:11 Antwort: 421 No transfer timeout (300 seconds): closing control connection
21:34:11 Fehler: Konnte vom Socket nicht lesen: ECONNRESET - Verbindung durch Peer zurückgesetzt (could not read from socket...)
21:34:11 Fehler: Verbindung zum Server getrennt (= connection closed)
Re: Filezilla stalls during upload
Posted: 2019-03-29 23:36
by botg
21:34:11 Antwort: 421 No transfer timeout (300 seconds): closing control connection
The server has closed an idle browsing connection through which in the last 300 seconds no transfers have been done.
On the other hand, the REAL issue seems to be: Why does the transmission stall periodically in the first place?
A log showing this would be useful.
the documentation on compiling from source is incomplete. It mentions ".configure"
Where did you find this? It must be a typo, it should refer to ./configure
Google has found malware on the Filezilla site
False-positive, their artificially intelligence has been running amok. Took their human maintenance staff 10 hours to find right plug to pull.
Re: Filezilla stalls during upload
Posted: 2019-03-30 10:36
by hstoellinger
Hello again,
(1) "Timeout": I know...
(2) "No transfers": correct! I disabled my iptables firewall just to make sure. No change!
(3) will produce a log this afternoon
(4) Yes, a typo on my part! However, there is NO configure anywhere within the directory path of the files I downloaded from the FileZilla download page.
Thanks again for your great help
H. S.
Re: Filezilla stalls during upload
Posted: 2019-03-30 16:22
by hstoellinger
Hello again,
Just finished a "LOOONG" upload!
As always -- after SOME of the "receiving contents of directory xyz" - messages from the server NOTHING happens. I can see this on the system monitor (KDE Sysmon) for both CPU load and network traffic. "Everything" just sits there, sometimes for UNDER 300 seconds (idle timeout); in which case transfer is resumed after - say - 50 seconds, sometimes the server disconnects because of the idle-timeout most likely specified on the server side. I simply wonder what causes this "waiting" and I am beginning to think that it's really the server who is the culprit.
By the way - these "stalls" also happen when trying transfers using lftp.
Here is the log. The transfer went to the end --- in the end!
Regards
H. S.
.... prior activity ...
17:02:06 Status: Dateiübertragung erfolgreich, 39,4 KB in 1 Sekunde übertragen || transfer successful
17:02:15 Befehl: PASV
17:02:15 Antwort: 227 Entering Passive Mode (213,145,224,140,195,175).
17:02:15 Befehl: STOR ExtensionTest.php
17:02:15 Antwort: 150 Opening ASCII mode data connection for ExtensionTest.php
17:02:15 Fehler: Verbindung vom Server geschlossen
17:02:15 Fehler: Dateiübertragung fehlgeschlagen nach Übertragung von 2,8 KB in 300 Sekunden || transfer unsuccessful
17:02:15 Status: Starte Upload von /home/hs/www/core/modules/system/tests/src/Kernel/Plugin/migrate/source/ExtensionTest.php
17:02:15 Status: Dateiübertragung erfolgreich, 792 B in 1 Sekunde übertragen
17:02:16 Status: Empfange Verzeichnisinhalt für "/www_8613/core/core/modules/system/tests/src/Kernel/Plugin/migrate/source"... || receiving directory content
17:02:16 Status: Dateiübertragung erfolgreich, 2,8 KB in 1 Sekunde übertragen
17:02:33 Befehl: TYPE I
17:02:33 Antwort: 200 Type set to I
17:02:33 Befehl: PASV
17:02:33 Antwort: 227 Entering Passive Mode (213,145,224,140,195,165).
17:02:33 Befehl: LIST -a
17:02:33 Fehler: Verbindung vom Server geschlossen || connection closed by server
17:02:33 Fehler: Dateiübertragung fehlgeschlagen
17:02:33 Status: Starte Upload von /home/hs/www/core/profiles/demo_umami/themes/umami/css/components/blocks/branding/branding.css
17:02:33 Status: Empfange Verzeichnisinhalt für "/www_8613/core/core/profiles/demo_umami/themes/umami/css/components/blocks/branding"...
17:02:33 Status: Dateiübertragung erfolgreich, 517 B in 1 Sekunde übertragen
17:02:36 Status: Sende Verbindungserhaltungs-Befehl || sending "keep-alive" message
17:02:45 Status: Sende Verbindungserhaltungs-Befehl
17:03:03 Status: Sende Verbindungserhaltungs-Befehl
17:03:06 Status: Sende Verbindungserhaltungs-Befehl
17:03:06 Status: Verbindung zum Server getrennt
17:03:15 Status: Sende Verbindungserhaltungs-Befehl
17:03:16 Status: Verbindung zum Server getrennt
17:03:33 Status: Sende Verbindungserhaltungs-Befehl
17:03:33 Status: Verbindung zum Server getrennt
17:06:21 Antwort: 421 No transfer timeout (300 seconds): closing control connection !!!!!
17:06:21 Fehler: Verbindung vom Server geschlossen
17:06:21 Fehler: Dateiübertragung fehlgeschlagen
17:06:21 Status: Verbindung zum Server getrennt
17:06:21 Status: Auflösen der IP-Adresse für rainer-regimentsmuseum-salzburg.at
17:06:21 Status: Verbinde mit 213.145.224.140:21...
17:06:21 Status: Verbindung hergestellt, warte auf Willkommensnachricht...
17:06:21 Status: Angemeldet
17:06:21 Status: Starte Upload von /home/hs/www/core/modules/file/tests/src/Unit/Plugin/migrate/cckfield/d7/FileCckTest.php
17:06:21 Status: Empfange Verzeichnisinhalt für "/www_8613/core/core/modules/file/tests/src/Unit/Plugin/migrate/cckfield/d7"...
17:06:22 Befehl: TYPE I
17:06:22 Antwort: 200 Type set to I
17:06:22 Befehl: PASV
17:06:22 Antwort: 227 Entering Passive Mode (213,145,224,140,195,175).
17:06:22 Befehl: LIST -a
17:06:22 Fehler: Verbindung vom Server geschlossen
17:06:22 Fehler: Dateiübertragung fehlgeschlagen nach Übertragung von 221 B in 300 Sekunden
17:06:22 Status: Verbindung zum Server getrennt
17:06:22 Status: Auflösen der IP-Adresse für rainer-regimentsmuseum-salzburg.at
17:06:22 Status: Verbinde mit 213.145.224.140:21...
17:06:22 Status: Verbindung hergestellt, warte auf Willkommensnachricht...
17:06:22 Status: Dateiübertragung erfolgreich, 2,6 KB in 1 Sekunde übertragen
17:06:22 Status: Angemeldet
17:06:22 Status: Starte Upload von /home/hs/www/core/modules/file/tests/src/Unit/Plugin/migrate/cckfield/d7/ImageCckTest.php
17:06:22 Status: Dateiübertragung erfolgreich, 1,9 KB in 1 Sekunde übertragen
17:06:22 Befehl: PASV
17:06:22 Antwort: 227 Entering Passive Mode (213,145,224,140,195,110).
17:06:22 Befehl: STOR FileTest.php
17:06:22 Fehler: Verbindung vom Server geschlossen
17:06:22 Fehler: Dateiübertragung fehlgeschlagen nach Übertragung von 2,4 KB in 300 Sekunden
17:06:22 Status: Starte Upload von /home/hs/www/core/modules/file/tests/src/Kernel/Plugin/migrate/source/d6/FileTest.php
17:06:22 Status: Empfange Verzeichnisinhalt für "/www_8613/core/core/modules/file/tests/src/Kernel/Plugin/migrate/source/d6"...
17:06:22 Status: Dateiübertragung erfolgreich, 2,4 KB in 1 Sekunde übertragen
17:06:40 Befehl: PASV
17:06:40 Antwort: 227 Entering Passive Mode (213,145,224,140,195,162).
17:06:40 Befehl: STOR FieldWidgetConstraint.php
17:06:40 Fehler: Verbindung vom Server geschlossen
17:06:40 Fehler: Dateiübertragung fehlgeschlagen
17:06:40 Status: Starte Upload von /home/hs/www/core/modules/system/tests/modules/entity_test/src/Plugin/Validation/Constraint/FieldWidgetConstraint.php
17:06:40 Status: Empfange Verzeichnisinhalt für "/www_8613/core/core/modules/system/tests/modules/entity_test/src/Plugin/Validation/Constraint"...
17:06:40 Status: Dateiübertragung erfolgreich, 401 B in 1 Sekunde übertragen
17:06:41 Status: Empfange Verzeichnisinhalt für "/www_8613/themes"...
17:06:41 Status: Anzeigen des Verzeichnisinhalts für "/www_8613/themes" abgeschlossen
17:06:52 Status: Sende Verbindungserhaltungs-Befehl
17:07:11 Status: Sende Verbindungserhaltungs-Befehl
17:07:22 Status: Sende Verbindungserhaltungs-Befehl
17:07:22 Status: Verbindung zum Server getrennt
17:07:41 Status: Verbindung zum Server getrennt
Re: Filezilla stalls during upload
Posted: 2019-03-31 12:11
by botg
Unfortunately the log is cluttered with unrelated content. Please configure FileZilla to only use on simultaneous transfer. After adding all files to the queue, close the browsing connection and only then start processing the queue. That way the log only contains the single transfer connection.
Re: Filezilla stalls during upload
Posted: 2019-03-31 20:26
by hstoellinger
Thanks again!
I have now set the number of concurrent connections and also upload transfers to 1. Now things seem to be fine! Seeing as FTP always uses two connections I still have somewhat of a "blank spot" about how the number of connections can be set to only one. Or, does this mean - number of connections to different servers? In any case - I am a happy boy now.
Thanks a lot
H. S.
Re: Filezilla stalls during upload
Posted: 2019-04-01 02:17
by boco
The concurrent connections you set in the Site Manager do, of course, only apply to the entries you set it for. The things you set in the global settings apply to all servers combined.