Discussion topic: It's the server's fault!

[botg]

The issue here is that there are two RFCs that describe two distinct methods of closing the TLS connection depending on how the TLS connection was established.

That's not correct. In all cases it is also the one and only TLS RFC that governs the shutdown behaviour, the FTP over TLS RFC cannot possibly influence it.

[da chicken]

Ah, I see. So the developers are implementing RFC 4217 when they should be implementing RFC 4346. So RFC 4217 is in error in it's example.

Have you notified IETF of this error?

[botg]

The examples are correct, they properly cover useless CCC command and file uploads. There just is no example for uploads. And examples are just that, examples. The details are in formal specs.

The problem is that some server developers misunderstood the concept of examples and applied the upload example to downloads.

This gets further complicated by the fact that OpenSSL is badly documented and broken in that it does not require a proper shutdown and doesn't make it easy to properly shutdown a connection.

[alliZ]

Send a bug report to SurgeFTP. Tell them the exact problem being caused. Give them links to this thread.

There's a problem with this suggestion:

On the two examples I gave, both servers are running the very same servers and versions (SurgeFTP ver. 2.3a3).

Given that we're running the very same servers and same server versions, and given that their success or failure only occurs immediately after:

08:37:57 Trace: CFtpControlSocket::ListSubcommandResult() <------------

... not at:

08:37:59 Trace: GnuTLS error -9: A TLS packet with unexpected length was received.
08:37:59 Status: Server did not properly shut down TLS connection
08:37:59 Error: Could not read from transfer socket: ECONNABORTED - Connection aborted

.. the exact problem being caused is therefore, still not known.

Could there not be alternative solutions, such as adjusting the server configuration or something else along those lines? Does any of this make sense?

!?!?

[stratos]

Hello, I have been following this thread as best as I can, though not being a developer makes it harder to understand all thats going on.

I am running FileZilla server version 0.9.27 beta
I am running FileZilla client version 3.1.5

I am getting the mentioned

Code: Select all

Status:	Server did not properly shut down TLS connection
Error:	Transfer connection interrupted: ECONNABORTED - Connection aborted
Error:	Failed to retrieve directory listing

It only happens on one directory on the entire ftp. Did I configure something incorrectly? I appreciate you taking the time to read this.

Regards,
Dave

P.S. I really like the FileZilla software, both client and server, thank you for writing it.

[da chicken]

Is there anything unique about this directory? Path name is long? Very large number of files?

Can you post complete logs instead of just the error message?

[Keylan]

I just had this same issue occur in one specific directory, and it looks like it was caused by a .DS_Store file being in the directory. Just a heads up.

[-Alex-]

I don't normally setup FTP servers (SFTP with SSH is good enough), but I've recently had to setup an FTP server due client needs. I was planning on trying to secure the server with TLS - but this is an absolute pain.

The problem I see with this is that you've treated the severity of this as fairly serious, with a security advisory and refusing to back down. Most of the FTP servers have treated this as a bug, an interoperability issue which needed fixing. The problem is that none of the Linux distributions have updated their FTP server packages because of it, unless it's treated as a proper security vulnerability I doubt they'll update.

Any chance of working with the creators of these FTP servers to get them to announce proper security advisories for this issue?

[KenK]

I've been getting this problem trying to list a directory when connected to an IBM z/OS server. The server has the IBM PTFs for RFC 4217 installed and the option to use it is set. Having read this thread, I raised it with IBM as a server problem. They analysed server and packet traces of the problem, and this is the response

When the passive mode FTP data connection is set up, we do the TLS negotiation, no errors occur - we then send 17 packets of 1460 bytes plus 1 packet of 390 bytes = 25210 bytes for the directory listing output.

All of these outbound packets are acknowledged by the FTP client - therefore the data for the directory listing should be at the client.

In addition to this, we do our shutdown() and close() calls, resulting in the FTP data session being taken down by an outbound FIN to the client, which the client acknowledges and send back their FIN, we then acknowledge this and the session is formally closed.

It is interesting that the directory listing was not displayed on the client especially when the data for this was sent to ( and subsequently acknowledged by ) the client TCP/IP stack.

I imagine this is because the client throws the following error :-
'GnuTLS error -9: A TLS packet with unexpected length was received.'

They're stuck there, they can't see the server doing anything wrong, and have asked for more information on why the client thinks it received a bad packet.

Trace from the client with Verbose debug and show raw directory listing on shows

Code: Select all

10:59:22	Command:	PASV
10:59:22	Trace:	CFtpControlSocket::OnReceive()
10:59:22	Response:	227 Entering Passive Mode (10,255,239,73,9,32)
10:59:22	Trace:	CFtpControlSocket::TransferParseResponse()
10:59:22	Trace:	CFtpControlSocket::SendNextCommand()
10:59:22	Trace:	CFtpControlSocket::TransferSend()
10:59:22	Command:	LIST
10:59:22	Trace:	CTransferSocket::OnConnect
10:59:22	Trace:	CTlsSocket::Handshake()
10:59:22	Trace:	CTlsSocket::Handshake()
10:59:22	Trace:	CTlsSocket::Handshake()
10:59:22	Trace:	CFtpControlSocket::OnReceive()
10:59:22	Response:	125 List started OK
10:59:22	Trace:	CFtpControlSocket::TransferParseResponse()
10:59:22	Trace:	CFtpControlSocket::SendNextCommand()
10:59:22	Trace:	CFtpControlSocket::TransferSend()
10:59:22	Trace:	CTlsSocket::Handshake()
10:59:22	Trace:	Handshake successful
10:59:22	Trace:	Session resumed
10:59:22	Trace:	Cipher: 3DES-CBC, MAC: SHA1
10:59:22	Trace:	CTransferSocket::OnConnect
10:59:23	Trace:	CTlsSocket::OnSocketEvent(): pending data, postponing close event
10:59:23	Listing:	Volume Unit    Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
10:59:23	Listing:	TSOD93 3390   2008/08/15  1   90  FB      80  3120  PO  AOC.COMBINED.ACF
... (entire directory listing as expected) ...
10:59:24	Listing:	TSOD92 3390   2008/07/18  3   60  VBA    137 27998  PS  ZOS18.PTFS.TXT
10:59:24	Trace:	GnuTLS error -9: A TLS packet with unexpected length was received.
10:59:24	Status:	Server did not properly shut down TLS connection
10:59:24	Error:	Could not read from transfer socket: ECONNABORTED - Connection aborted
10:59:24	Trace:	CTransferSocket::TransferEnd(3)
10:59:24	Trace:	CFtpControlSocket::TransferEnd()
10:59:24	Trace:	CFtpControlSocket::OnReceive()
10:59:24	Response:	250 List completed successfully.
10:59:24	Trace:	CFtpControlSocket::TransferParseResponse()
10:59:24	Trace:	CFtpControlSocket::ResetOperation(2)
10:59:24	Trace:	CControlSocket::ResetOperation(2)
10:59:24	Trace:	CFtpControlSocket::ParseSubcommandResult(2)
10:59:24	Trace:	CFtpControlSocket::ListSubcommandResult()
10:59:24	Trace:	CFtpControlSocket::ResetOperation(2)
10:59:24	Trace:	CControlSocket::ResetOperation(2)
10:59:24	Error:	Failed to retrieve directory listing

Is there any way to get more information on why this happens:
GnuTLS error -9: A TLS packet with unexpected length was received.

Regards, Ken

Edit: above trace from Windows 3.1.5 client. Just tried again using 3.1.6 - no change. z/OS system is 1.8.

[botg]

Having read this thread, I raised it with IBM as a server problem. They analysed server and packet traces of the problem, and this is the response

All of these outbound packets are acknowledged by the FTP client - therefore the data for the directory listing should be at the client.

Wrong. They get acknowledged by either the client or a man in the middle. SSL only protects the data, not the TCP headers. If you get an ACK, it might as well come from somebody else.

In addition to this, we do our shutdown() and close() calls, resulting in the FTP data session being taken down by an outbound FIN to the client, which the client acknowledges and send back their FIN, we then acknowledge this and the session is formally closed.

Again, wrong. Somebody acknowledges the FIN, not necessarily the client, could as well have been a man in the middle.

Before the shutdown() and the close(), you have to perform an orderly TLS/SSL shutdown as described in section 7.2.1. or RFC 2246.

If the client gets a FIN before a closure alert, it has to assume the FIN comes from an attacker and fail the connection.

If IBM is unable to follow some simple RFCs, you might want to switch to a more experienced vendor, as IBM apparently isn't worth its money.

[CrazyGolem]

Le serveur n'a pas fermé correctement la connection TLS
Erreur : Could not read from transfer socket: ECONNABORTED - Connection aborted
Réponse : 226 Closing data connection.
Erreur : Échec à la lecture du contenu du répertoire
Statut : Déconnecté du serveur

This looks like a quiet obscure message to the lambda user. This lambda user will see this and think : When I try to connect, it does not work. So I'll try another client and it will work fine.
And so, he will not mail the server admin because everything works fine now.

So, if you really don't want a "transitional fixing" scheme, at least add some informations to TELL the user that the server has a problem, and which kind of problem (e.g. : The server does not respect the communication scheme for TLS and it's a security leak, so FileZilla won't accept the connection).
So you'll be almost sure that the user will tell the server admin about this problem ("My FTP client told me that the server has a security leak" and not "My client don't work with your server"). THIS will make developers and/or the server admin do something, not the obscure message that is displayed now.

PS : As you probably saw, English is not my mother language, so I apologize for mistakes I could have made

[KenK]

I've been getting this problem trying to list a directory when connected to an IBM z/OS server.

See PK77240

[phillyidol]

Hey there,

I found a weird anomoly that I thought had something to do with this thread (it still may), and I was able to fix it, but it was just a little weird as to how. First off, I was getting that same "ECONNABORTED - Connection aborted" error, so I made sure I had the most current FileZilla client, and FileZilla Server on my server (BTW - running Windows Server 2003 SE).

Was happily FTP'ing up to my site, with SSL settings (so FTPS), when I added one additional img file to my images folder, then - bam - error showed up , but only for that folder - no others. Started checking thru forums to see what it referred to and voila - here I be.

Anyway, after realizing (from this thread) that it certainly wasn't FileZilla's fault - I tried a test. I had exactly fifty-eight (58) image files in the images folder (even mix of .jpg, .gif, and .png), and two (2) other sub-folders named 'en' and 'fr' (english and french respectively).

I went onto my server, removed one of the .jpg files - error went away, put it back - error came back . Removed one of the .gif files - error went away, put it back - error came back. Removed one of the .png files - error went away, put it back - error came back. Then, removed one of the sub-folders (tried it with both) - error went away, put it back - error came back (tried it with both). It didn't matter what files I chose, traded, uploaded, (I even tried re-naming), whatever - as soon as there was 2 sub-folders, and 58 img files, the error showed up (I tried adding document files - .doc, .txt, etc. - same result).

Then I tried adding to see what happened, and if I only added one (1) more file (img or doc), the error remained, but if I added two (2) more files (again img, doc, or 1 of each) the error went away . So then, if I left the img files at 58, or 59, and added a third sub-folder - error went away (BTW - I even moved the entire images folder to the root of C:, and it still happened this way).

It seems the magic equation is '2 sub-folders + 58, or 59, files (img or doc) = error'. Basically, I'm very confused by why, but end result - I created a third sub-folder called 'temp', and havent seen the error since.

Anyway, if there's anyone who has any thoughts on this, please feel free, as I'm very curious about it.

Cheers

P.S. To 'botg' (or Tim, if I may) you sir are a lifesaver with these programs, and should be very proud of how well they work (I know I would be) . Thanks for both.

[stinkee]

Thank you philly for your entry. I read through 6 pages of RFC Discussion before getting to your post.

It's unfortunate that someone using the latest versions of both the Filezilla server and the Filezilla client are receiving the ECONN message when trying to list files.

I can attest that the fix that Philly stated works. I created a directory called blank in the folder the user was trying to access and then the client connected properly. I hope attention is given to this issue apart from the RFC discussion.

Thanks!

[phillyidol]

Hey there,

No worries to all, but I realized I never gave an example for anyone to view/test with (my bad).

I turned on debugging on my FZ client, AWA set it to give out the raw directory data, and reconnected to the folder. Here's the raw info below with 2 directories, and 58 img files. Plus, I highlighted the errors . Hope this is useful for testing, or a fix - enjoy:

(Just FYI - I renamed one of my directories for a test, so instead of my original 2 'en' + 'fr' - they're 'fr' & 'resized' in the below)

CFtpControlSocket::TransferParseResponse()
14:50:04 Trace: code = 2
14:50:04 Trace: state = 5
14:50:04 Trace: CFtpControlSocket::SendNextCommand()
14:50:04 Trace: CFtpControlSocket::TransferSend()
14:50:04 Trace: state = 8
14:50:04 Trace: CTlsSocket::OnRead()
14:50:04 Trace: CTransferSocket::OnReceive(), m_transferMode=0
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 74 Jan 12 14:48 bg-columnbox.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 150 Jan 12 14:48 bg-columnbox.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 271 Jan 12 14:48 bg-footer.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 279 Jan 12 14:48 bg-footer.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 8447 Jan 12 14:48 bg-inthenews.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5179 Jan 12 14:48 bg-inthenews.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 278 Jan 12 14:48 bg-main-end.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 293 Jan 12 14:48 bg-main-end.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 123 Jan 12 14:48 bg-main.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 174 Jan 12 14:48 bg-main.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 12948 Jan 12 14:48 bg-menu.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 4239 Jan 12 14:48 bg-menu.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 26299 Jan 12 14:48 bg-page.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 11234 Jan 12 14:48 bg-search.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 3512 Jan 12 14:48 bg-search.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 789 Jan 12 14:48 bg-stockticker-end.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 572 Jan 12 14:48 bg-stockticker-end.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 2979 Jan 12 14:48 bg-stockticker.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 3297 Jan 12 14:48 bg-stockticker.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 6349 Jan 12 14:48 bg-TSN-sports.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 9123 Jan 12 14:48 bg-TSN-sports.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 1356 Jan 12 14:48 bg-TSNend.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 573 Jan 12 14:48 bg-TSNend.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 49 Jan 12 14:48 blank.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 26012 Jan 12 14:48 bobs-blog.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 1224 Jan 12 14:48 button-go.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 229 Jan 12 14:48 columnbox-end.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 259 Jan 12 14:48 columnbox-end.png
14:50:04 Listing: drwxr-xr-x 1 ftp ftp 0 Jan 12 14:49 fr
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 164 Jan 12 14:48 hr.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 2872 Jan 12 14:48 hr.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 20949 Jan 12 14:48 large-photo.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 29730 Jan 12 14:48 large-photo.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 16333 Jan 12 14:48 logo-forzani-group.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 13871 Jan 12 14:48 logo-forzani-group.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 2938 Jan 12 14:48 logo-sportchek.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 8818 Jan 12 14:48 logo-workforz.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 11691 Jan 12 14:48 logo-workforz.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 61 Jan 12 14:48 nav-arrow.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 70 Jan 12 14:48 nav-list-bullet.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 62 Jan 12 14:48 nav-list-closed.gif
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 60 Jan 12 14:48 nav-list-open.gif
14:50:04 Listing: drwxr-xr-x 1 ftp ftp 0 Jan 12 14:49 resized
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 689 Jan 12 14:48 rss-icon.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5082 Jan 12 14:48 small-photo.jpg
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 7408 Jan 12 14:48 small-photo.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5341 Jan 12 14:48 titlebar-articlearchive.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5469 Jan 12 14:48 titlebar-banners-franchises.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5722 Jan 12 14:48 titlebar-bobsblog.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5720 Jan 12 14:48 titlebar-calendar.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5467 Jan 12 14:48 titlebar-dailypoll.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5407 Jan 12 14:48 titlebar-forms-tools.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5092 Jan 12 14:48 titlebar-mycareer.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 6096 Jan 12 14:48 titlebar-onthemove.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5354 Jan 12 14:48 titlebar-ourcommunity.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 5241 Jan 12 14:48 titlebar-ourcompany.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 2451 Jan 12 14:48 titlebar-ourpeople.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 6350 Jan 12 14:48 titlebar-welcome-FGL.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 6074 Jan 12 14:48 titlebar-wellnesstip.png
14:50:04 Listing: -rw-r--r-- 1 ftp ftp 1206 Jan 12 14:48 titlebar.jpg
14:50:04 Trace: CTlsSocket::OnRead()
14:50:04 Trace: GnuTLS error -9: A TLS packet with unexpected length was received.
14:50:04 Status: Server did not properly shut down TLS connection
14:50:04 Trace: CTlsSocket::OnSocketEvent(): close event received
14:50:04 Trace: CTransferSocket::OnClose(10053)
14:50:04 Error: Transfer connection interrupted: ECONNABORTED - Connection aborted
14:50:04 Trace: CTransferSocket::TransferEnd(3)
14:50:04 Trace: CFtpControlSocket::TransferEnd()
14:50:04 Trace: CFtpControlSocket::ResetOperation(2)
14:50:04 Trace: CControlSocket::ResetOperation(2)
14:50:04 Trace: CFtpControlSocket::ParseSubcommandResult(2)
14:50:04 Trace: CFtpControlSocket::ListSubcommandResult()
14:50:04 Trace: state = 3
14:50:04 Trace: CFtpControlSocket::ResetOperation(2)
14:50:04 Trace: CControlSocket::ResetOperation(2)
14:50:04 Error: Failed to retrieve directory listing


P.S. As far as the error discussion goes, I agree that the RFC is important, and I do understand that not properly following the "correct" rules (especially for security), is a very big deal (not that anyone has really said it wasn't - just adding my 2c), so I'm on board with that. This was just such a weird anomoly, and I didn't know if it was connected in any way (I'm certainly NO guru), but thought this may be a alternative issue to look at as to why the error is there in some cases, when you do have the most current FZ Client and Server installed, so I figured I'd post it JIC.

Other than that, Cheers to all.

Phillyidol