IIS 7.5 FTPeS: unexpected TLS packet length: Varies by cert

[AndrewS]

Hello,

We are having a strange issue where some of our users connecting to IIS 7.5 FTP with Explicit TLS are receiving the dreaded "GnuTLS error -9: A TLS packet with unexpected length was received. / Server did not properly shut down TLS connection" error message. This only happens using the FileZilla Client -- all other FTP clients we have tested work.

I have already read through all of the existing threads on the forum relating to this issue, but we are seeing something unique: On the same server, same instance of IIS, the TLS negotiation succeeds with some SSL certificates but fails for others. I have also eliminated the other common trouble shooting tasks: a) symptoms occur from remote clients AND while testing on localhost, therefore, it is not a firewall issue. b) certificate are properly installed

For example (go ahead and test these using anonymous authentication):

173.165.230.67:21 uses *.pc-one.net issued by an internal/private CA. TLS negotiation succeeds.

173.165.230.68:21 uses *.pc-one.net issued by DigiCert (US). TLS negotiation fails.

As I said, both of these FTP sites are on the same server. If I switch the SSL certificate bindings between the two sites, the same results are obtained based on which certificate is bound.

Notable difference between the certs:
- Internal cert has no intermediate CAs with signature algorithm: sha512RSA
- External cert has 2 intermediate CAs with signature algorithm: sha1RSA

Since the external cert has 2 additional certificates in the hierarchy, I am wondering if that additional size is the issue. Meaning, the data is not being broken up into packets correctly or not being reconstructed correctly?

I look forward to your help. Thank you!
Andy

[botg]

Please post comparative logs with debug level set to 4.

[AndrewS]

Hello botg,

Thank you for the quick reply. I have attached two text files with the logs from both connections.

"success.txt" is from 173.165.230.67:21 uses *.pc-one.net issued by an internal/private CA.

"failure.txt" is form 173.165.230.68:21 uses *.pc-one.net issued by DigiCert (US).

[botg]

Not a bug in FileZilla, it also happens with the gnutls-cli tool which is part of the GnuTLS library used by FileZilla. The handshake aborts before even being offered the certificate.

Three possibilities:
- Something weird about the certificate which prevents the server from even offering it to the client
- A bug in your server.
- A bug in GnuTLS

Considering the handshake aborts before being offered the certificate, option 3 seems very unlikely.

[AndrewS]

Thank you for checking into this, I appreciate it. I agree that is it not a bug in FileZilla.

I have done some additional testing of certificates and have come to the conclusion that the first option seems the most likely. I have tried different certificates issued by the same CA (DigiCert US) and they work fine. So that leads me to think that something peculiar about the certificate is causing the handshake between IIS FTP and GnuTLS to fail. Interestingly enough, the handshake succeeds for other clients, such as FlashFXP which uses OpenSSL.

I'll take this issue up with MSFT and the GnuTLS mailing lists.

Thanks again,
Andy

[heitjr]

I think I might be able to add something to this. FileZilla (using GnuTLS) sends a TLSv1.2 ClientHello. TLSv1.2 requires servers to handle the Signature Algorithms extension of Hello messages, and clients to send it if they want anything other than the default (RFC 5246). The ClientHello that is being sent by FileZilla (using GnuTLS) includes only one Signature Algorithm as defined by the RFC. The algorithm is {SHA512,RSA}. This indicates to the server that the client only wishes to be presented signatures using SHA512/RSA. A bit restrictive. If the server does not support TLSv1.2, all is fine as it ignores the extension. if the server does support TLSv1.2 and has a SHA1/RSA certificate....well we've got a problem....See possiblility #1 above.

I approached this as a GnuTLS problem, and reported this to the GnuTLS developers. Here is the text of my e-mail, and the text of the response:

Me:

"Hello,
I have been working on the implementation of the TLS 1.2 protocol. TLS 1.2 requires servers to handle the Signature Algorithms extension to the ClientHello handshake message. My reading of RFC 5246 (7.4.1.4.1.) indicates that if client presents the extension (it can be omitted) it should include all hash/signature algorithm pairs the client is willing to process. While running the latest version of FileZilla, which uses GnuTLS 2.10.4, the only proposed Signature Algorithm is {SHA512,RSA}. If I stick with the RFC, the handshake will fail, as my {SHA1,RSA} signed certificate is not in the list.
I’m not saying Microsoft is always right (in this case I think they are), but IE8/Win7 sends 7 Signature Algorithms in the extension: {SHA256,RSA},{SHA384,RSA},{SHA1,RSA},{SHA256,ECDSA},{SHA384,ECDSA},{SHA1,ECDSA},{SHA1,DSA}."

Response from GnuTLS:

"Hello,
This is a configuration issue. Filezilla for some reason unknown to me only enables 256-bit ciphersuites and signature algorithms. If you use gnutls-cli with your server you'll see that gnutls sends all options."

So, GnuTLS is pointing to FileZilla as the source of this defect. I will be submitting a ticket.

[botg]

That's peculiar, SHA256 and higher should be enabled. SHA1 on the other hand has been disabled on purpose as many weaknesses have already been discovered in SHA1 with full breakage likely coming soon.

[heitjr]

Full breakage soon? All SSL/TLS versions prior to V1.2 use SHA1 for HMAC, and almost all SSL/TLS traffic is prior to V1.2. Most certificates in circulation still use SHA1 based signatures? If FileZilla Client will not support SHA1 based signatures when using TLSV1.2, I think there could be some severe interoperability problems when people start deploying TLSv1.2 servers. I may need to code my TLSv1.2 handshake processing to fall back to TLSv1.1 if the client specifies signature algorithms that do not include my certificate---I'm not sure if Dierks and Rescorla (TLSv 1.2 RFC 5246 authors) would be in favor of that.....Thanks for the quick response. I have submitted a ticket---we will see how that goes......