FileZilla 3 development diary

[botg]

In the next version of FileZilla the following FTP over TLS changes will be made:

• Support for the insecure RC4 algorithm will be removed
• The certificate verification dialog will show subject alternative names
• When a TLS alert is received, it now says which alert got received
• Improved error reporting on underlying socket instead of the generic "error in the push/pull function"

In some future version, the following changes will be made:

• Mismatch of server hostname with the certificate's common name or subject alternate name will become an error.
• Session reuse on the data connection will be mandatory

[botg]

Ever since 3.10 came out, there has been an issue with the progress bar in the queue updating very sluggishly when using SFTP in combination with speed limits. It's a complex issue to fix: What broke the progress bar updates actually greatly increases SFTP performance when no limits are set. Fixing it so that the speed remains great if no limit is set and the progress bar updates quickly if a limit is set requires extensive infrastructure changes.

What essentially happens is this: Data is uploaded in chunks. Once the server has received the chunk it sends an acknowledgement back. FZ updates its transfer each time an acknowledgement arrives. These chunks are quite big. With speed limits set to a low value, it takes a long time to transfer a chunk and the progress is thus rarely updated. In 3.10 chunk size has been increased significantly to match the bandwidth-delay product of the underlying TCP connection.

To solve this, my idea is to dynamically adjust the chunk size based on the currently set speed limit and potentially also the current transfer speed (as on very slow network connections the progress update can be slow as well).

For this to work, the current speed limit needs to be known by fzsftp.exe which handles the SFTP connection. Currently fzsftp has no idea about the actual limit, it only knows its own remaining quota in the current timeslice. (FZ does speed limits with token buckets; There's one bucket per connection, new tokens every 250ms)

To inform fzsftp about the current limit in an efficient manner, my idea is to send a command to fzsftp each time the limit changes.

However, while there already is code in FileZilla to inform some of its parts of option changes, this code currently only works for the GUI thread, it cannot inform non-GUI threads. All FTP and SFTP processing is done in a worker thread though, hence I need to extend this code to work with the thread event loop.

The two main challenges in implementing this are a) to make it fast, if the limit never changes there must be zero overhead and to b) make it thread-safe in the context of event handlers that can get asynchronously deleted.

Another issue with the current code is that it calls the option changed callback once for each option changed. I want it to be called only once, with a list of changed options passed to the callback. An easy changed compared to the above, but still tedious to change the existing users.

So far trying to fix this one bug has already caused some fallback:
- Fixed a bug with the total transfer size being too small in the log after a successful SFTP upload
- Inefficiencies passing the current transfer status around in FileZilla
- Even if no limit was set, the token buckets were needlessly updated every 250ms
- The speed limit code wasn't even thread-safe

While not much code has been implemented to address the original bug yet, the hardest part is over: Conceptualizing a solution.

[botg]

RTTI in any language, C++ included, is slow.

If you have a simple type hierarchy though where you can change the base class and only ever need to cast to leaf classes, there's a simple trick you can use.

Here's a simple example showing how a holder for arbitrary types can be used for dynamic dispatch.

Code: Select all

#include <iostream>

struct holder_base {
    virtual ~holder_base() {}

    virtual char const* derived_type() const = 0;
};

template<typename T>
struct holder final : holder_base {
    holder() {}

    static char const* type() {
        static const char t = 0;
        return &t;
    }

    virtual char const* derived_type() const {
        return type();
    }

    T t_;
};

template<typename T>
bool is_type(holder_base const& b) {
    return b.derived_type() == holder<T>::type();
}


struct A{};
struct B{};

void stuff(A const& a)
{
    std::cerr << "Called with A\n";
}

void stuff(B const& b)
{
    std::cerr << "Called with B\n";
}

template<typename T> void do_dispatch(holder_base const &h)
{
    if (is_type<A>(h)) {
        stuff(static_cast<holder<T> const&>(h).t_);
    }
}

void dispatch(holder_base const& h) {
    do_dispatch<A>(h);
    do_dispatch<B>(h);
}

int main() {
    holder<A> foo;
    holder<B> bar;
    dispatch(foo);
    dispatch(bar);
    return 0;
}

[botg]

Some small quality of life changes are coming to FileZilla Server:

Connecting to server [::1]:14147...
Connected, waiting for authentication
Logged on
You appear to be behind a NAT router. Please configure the passive mode settings and forward a range of ports in your router.
Warning: FTP over TLS is not enabled, users cannot securely log in.

[boco]

Maybe a button or Help menu entry that leads to the https://ftptest.net page would be nice.

[botg]

A few words on Logjam:

FileZilla Client sadly accepts too small DH primes, I'll release 3.11.0.1 later this week to disallow tiny primes.

FileZilla Server on the other hand doesn't seem to suffer from Logjam for the simple fact that it doesn't seem to support DHE at all. The OpenSSL API makes it extremely difficult to use something simple as Diffie-Hellman, without first speaking some complex incantations when Saturn stands in the fifth house during harvest moon. Knowledge of the incantations is only passed on by divine enlightenment, you need to know them in order to learn them.

[botg]

On a related note, it annoys me endlessly that most online testers for TLS only seem to deal with HTTPS and not other protocols using TLS, e.g. FTPS or SMTPS.

Luckily the domains tlstest.org, tlstest.com and tlstest.net were still available. Now I've got another project to work on.

[botg]

Why do consumer-grade firewalls have such utterly shit quality?

First there's AVG which causes random corruption of transferred files, now there's Kaspersky that's not only preventing the data connection from becoming established, but also injects fraudulent TLS certificates into connections.

Expect FZ 3.11.0.2 in a couple of days, it'll blacklist Kaspersky's fake certificates.



Please tell me if you know of any other third-party malware that injects fake TLS certificates.

[boco]

Many antivirus scanners do that now. For example avast (to my knowledge it only filters HTTPS so far).

[botg]

Let's throw actual security out of the window in order to give the user a false sense of security.

Such a great idea. Nothing could possibly go wrong.

[boco]

First thing I did was *facepalm*.

Second thing was to disable that "feature".

[ajuaristi]

Hi all,

Recently I scheduled a new patch for FileZilla implementing a per-host mapping to PPK keys: viewtopic.php?f=3&t=37073

Following Tim Kosse's feedback, I've already done my first tweaks at the code, and want to keep you informed. This is how it looks now under Linux:



It does not do anything for now, but that'll come later.

Feel free to drop your feedback.

[botg]

Where can I type in the filename directly?

The password field isn't needed here, as the aim is public key authentication.

What about this?

keyfile.png (62.76 KiB) Viewed 45082 times

[ajuaristi]

Hi,

It's a good idea, but if I did that, the public key list on "Edit -> Settings -> SFTP" would not be needed anymore. In my example screenshot, the "Key file" drop-down would be filled with the PPKs added by the user in that list, so that you can select one of those.

I recall, it could be done your way, but the section "Edit -> Settings -> SFTP" would no longer be needed.

What do you think?

[botg]

It's a good idea, but if I did that, the public key list on "Edit -> Settings -> SFTP" would not be needed anymore.

The list of keys in the settings would be used if there is no specific key given. Very useful feature for most users that only have one SSH private key and use it on all their servers, that way they do not have to create Site Manager entries. Users having a single SSH private key, with servers having a potentially large list of trusted public keys is SSHs preferred mode of operation after all, as is evident from the design of the OpenSSH reference implementation.

The primary use-case for the site-specific key on the other hand would be for users that have a tremendous amount of private keys and furthermore connect to servers that artificially limit the amount of keys the client can present. A complete inversion of the intended design.


Use-cases aside and into the realm of implementation details: In FZ, sites and settings are different entities, there is no coupling. Having a site depend on a setting (ie. the configured list of keyfiles) would introduce all sorts of new error case to be handled. Just some of the examples that immediately come to my mind:
- What happens if changing a key from the keyfile list, should the Site Manager entry change?
- What happens if a Site Manager entry references a key that isn't stored in the settings?
- Two instances of FZ running, one uses the Site Manager to change the keyfile for a site, the other has the settings open to manage keyfiles. Click OK in both dialogs. The latter in particular has interesting consequences if one were attempting to solve it.