Connection Handling - Client constantly reconnects

[miketonks]

Filezill 3.0.3 & 3.0.2.1 - Windows 2000

I am finding with strange behaviour with ftp connections in Filezilla 3.x.

I can connect to server fine. When I transfer a file, sometimes it will use the open connection - this is good, it means a very quite transfer. Sometimes it opens a new connection - this is bad, it means I have to wait while dns and authentication process is completed again, and seems unnecessary.

Below are 3 example log snippets which demonstrate this behaviour.

In the first example, I connect to upload a file, then upload the file again after a minor code change (debugging). See how filezilla connects three times.

The second example demonstrates a slightly different pattern. Between uploads, filezilla client disconnects. Again - unwanted behaviour.

The third example demonstrates another pattern - client disconnects with a timeout message. Note that it does this regardless of the timeout settings in filezilla client - I have tried values of 0 and 9999 and neither alters this behaviour.

Note that I have tried various combinations of settings -> transfers -> max simultaneous transfers // timeout in seconds and am unable to find a satisfactory solution.

When uploading a batch of files, simultaneous transfers is good, but when dealing with say two files, it is ungainly as it connects twice when one connection would be quicker. Preferably client should use the open connection always when available, and only open additional connections if say queue length is beyond a certain limit.

I should say that I have used many ftp clients over the years and never seen this problem before. Previous filezill versions (2.x? - sorry can't remember) did not do this.

I do not think this is a firewall issue. I have a linksys wrk54g and have never had any problems with it.

Log Excerpts
---------------

Status: Resolving IP-Address for spot.gn.apc.org
Status: Connecting to 217.72.179.10:21...
Status: Connection established, waiting for welcome message...
Response: 220 ProFTPD 1.3.0 Server (Debian) [217.72.179.10]
Command: USER comresolve
Response: 331 Password required for comresolve.
Command: PASS ********
Response: 230 User comresolve logged in.
Command: SYST
Response: 215 UNIX Type: L8
Command: FEAT
Response: 211-Features:
Response: MDTM
Response: REST STREAM
Response: SIZE
Response: 211 End
Status: Connected
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is current directory.
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,149,107).
Command: LIST
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,201,206).
Command: LIST -a
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.
Status: Calculating timezone offset of server...
Command: MDTM .forward
Response: 213 20071114132105
Status: Timezone offsets: Server: 0 seconds. Local: 0 seconds. Difference: 0 seconds.
Status: Directory listing successful
Status: Retrieving directory listing...
Command: CWD cgi-bin
Response: 250 CWD command successful
Command: PWD
Response: 257 "/cgi-bin" is current directory.
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,156,202).
Command: LIST -a
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.
Status: Directory listing successful
Status: Retrieving directory listing...
Command: CWD cms
Response: 250 CWD command successful
Command: PWD
Response: 257 "/cgi-bin/cms" is current directory.
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,215,60).
Command: LIST -a
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.
Status: Directory listing successful
Status: Resolving IP-Address for spot.gn.apc.org
Status: Connecting to 217.72.179.10:21...
Status: Connection established, waiting for welcome message...
Response: 220 ProFTPD 1.3.0 Server (Debian) [217.72.179.10]
Command: USER comresolve
Response: 331 Password required for comresolve.
Command: PASS ********
Response: 230 User comresolve logged in.
Status: Connected
Status: Starting upload of G:\mike\work\comresolve\secure\cgi\common_functions.pm
Command: CWD /cgi-bin/cms/
Response: 250 CWD command successful
Command: PWD
Response: 257 "/cgi-bin/cms" is current directory.
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,220,229).
Command: STOR common_functions.pm
Response: 150 Opening BINARY mode data connection for common_functions.pm
Response: 226 Transfer complete.
Status: File transfer successful
Status: Disconnected from server
Status: Resolving IP-Address for spot.gn.apc.org
Status: Connecting to 217.72.179.10:21...
Status: Connection established, waiting for welcome message...
Response: 220 ProFTPD 1.3.0 Server (Debian) [217.72.179.10]
Command: USER xxx
Response: 331 Password required for xxx .
Command: PASS ********
Response: 230 User xxx logged in.
Status: Connected
Status: Starting upload of G:\mike\work\comresolve\secure\cgi\common_functions.pm

...

Command: STOR common_functions.pm
Response: 150 Opening BINARY mode data connection for common_functions.pm
Response: 226 Transfer complete.
Status: File transfer successful
Status: Disconnected from server
Status: Resolving IP-Address for abc.gn.apc.org
Status: Connecting to 217.72.179.10:21...
Status: Connection established, waiting for welcome message...
Response: 220 ProFTPD 1.3.0 Server (Debian) [217.72.179.10]
Command: USER xxx
Response: 331 Password required for xxx.
Command: PASS ********
Response: 230 User xxx logged in.
Status: Connected
Status: Starting upload of G:\mike\work\xxx\secure\cgi\common_functions.pm
Command: CWD /cgi-bin/cms/
Response: 250 CWD command successful
Command: PWD
Response: 257 "/cgi-bin/cms" is current directory.
Status: Retrieving directory listing...
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,173,69).
Command: LIST -a
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.

...

Command: STOR common_functions.pm
Response: 150 Opening BINARY mode data connection for common_functions.pm
Response: 226 Transfer complete.
Status: File transfer successful
Status: Disconnected from server
Response: 421 No Transfer Timeout (300 seconds): closing control connection.
Error: Disconnected from server

...

[miketonks]

I should add that handling of the "Remote Site Browsing" window exhibits the same behaviour.

I can log in to the server and browse directories on the server. I then upload a file by either double click or drag drop, and the client opens a new connection for no apparent reason. Log file (below) says "Disconnected from Server". I can continue to browse the server without the existing connection.

...

Command: CWD /cgi-bin/twiki/
Response: 250 CWD command successful
Command: PWD
Response: 257 "/cgi-bin/twiki" is current directory.
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,194,248).
Command: LIST -a
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.
Status: Directory listing successful
Status: Retrieving directory listing...
Command: CWD /https/
Response: 250 CWD command successful
Command: PWD
Response: 257 "/https" is current directory.
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,176,119).
Command: LIST -a
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.
Status: Directory listing successful
Status: Disconnected from server
Status: Resolving IP-Address for abc.gn.apc.org
Status: Connecting to 217.72.179.10:21...
Status: Connection established, waiting for welcome message...
Response: 220 ProFTPD 1.3.0 Server (Debian) [217.72.179.10]
Command: USER xxx
Response: 331 Password required for xxx.
Command: PASS ********
Response: 230 User xxx logged in.
Status: Connected
Status: Starting upload of G:\mike\work\xxx\secure\cgi\common_functions.pm
Command: CWD /cgi-bin/cms/
Response: 250 CWD command successful
Command: PWD
Response: 257 "/cgi-bin/cms" is current directory.
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (217,72,179,10,166,18).
Command: STOR common_functions.pm
Response: 150 Opening BINARY mode data connection for common_functions.pm
Response: 226 Transfer complete.
Status: File transfer successful

[botg]

This is by design. FileZilla uses secondary connection to transfer files to ensure that you can still browse the server during transfers.

[miketonks]

Thanks - that makes sense. Has there been any consideration of keeping the file transfer connection open? It seems to stay for a short while, I'm not certain how long. I was expecting the settings value 'timeout in seconds' to affect this, but it does not seem to.

The particular usage example where I think this would be a big advantage is debugging files on a remote server with only ftp access. It's something I have to do quite often - upload files, check output / error log, edit file locally and then upload again, over and over until the problem is fixed. Often there will be many files to edit.

Editing a file in this way is sometimes within the current timeout limit, and sometimes not. When it is still connected, the file just whips across and the whole process is sweet. When it has to connect again, it's a pain - ok so probably my ftp server is slow with the login process, but it seems wrong for it to have to do this all the time.

Of course - there's another way to do this with the 'File Viewing / Editing' facility. This is great because it remembers where to put the file, and in the latest version does a great job of detecting changes to files being edited. However, it falls into the same problem that it constantly has to reconnect each time.

In both cases it would be really cool if the secondary connection for file transfers were kept open longer (configurable).

What would be doubly cool is if the 'File has changed' dialogue could have a 'always use this answer' option much like the overwrite files dialogue. That way, changes saved through the edit remote file facility would automatically push back to the server without needing to switch the the client window and ok the dialogue each time.

One more thing - the current timeout setting is definately not working, even on the primary connection. I get this after 5 minutes:

Response: 421 No Transfer Timeout (300 seconds): closing control connection.

but my config is set to 9999 seconds.

[boco]

For the last part (transfer timeout 300 secs):

This timeout limit is enforced by the server, the client can't do much about it. Activity on the server resets this timeout, as it is a transfer timeout, you actually neet to upload/download a file.

You could counter this with keep-alive, but Filezilla3 has none yet.

edit: The timeout in secs (0-9999) in Filezilla is there to deal with stale connections, it can't change the server-imposed limit.

[botg]

No Transfer Timeout cannot be countered by keep-alive.

[boco]

Sorry, my fault. Correction:

Keep-alive is useful against connection timeouts, but useless against no-transfer timeouts.

[AdrianasFirst]

My ISP hosts 200,000 sites and their FTP servers permit connections to be kept alive with periodic NOOP messages.

For me, this is FileZilla's biggest shortcoming, and prevents me from removing SmartFTP from my computer.

[boco]

Latest Filezilla has keep-alive, you must enable it manually.

[jesus2099]

Keep-alive is not effective with no-transfer timeouts.
What shall be done is like in SmartFTP :

Let the fileview displayed even if there has been a timeout (offline mode).
Reconnect on the fly when there is a new transfer.

ie. SmartFTP behaviour.

[jesus2099]

So no one is interested in this offline mode (smartFTP) that reconnects whenever needed (UL, DL, show a new filelist) while letting the display and navigation of already browsed folders possible ?

It is useful for no transfer timeouted servers.

I think that the "R" button is quite old-fashioned. Why not let the remote folder tree and filelists in place at timeout ?

[boco]

Did you even try out the latest version yet? Has been added in 3.1.3 already. Have fun.

[jesus2099]

Wow are you serious ?
I'll try that right away, thank you !

That's right, it is now working ! This is a major improvement !!

This is no blackmail but, this is letting only two things in my whishlist : auto-upload in file monitoring mode and simple remote explorer window.