Multithreaded Encryption for FTP Over TLS

[Particle]

Are there any options for multithreaded software encryption that can be enabled in FileZilla server? If not, I'd like to request it.

I recently upgraded a storage server of mine from a single Opteron 6234 to dual Opteron 6180 SE processors. The former is an Interlagos based processor (ie Bulldozer). The latter are Magny Cours based processors (ie Thuban). AMD introduced hardware AES acceleration in their chips starting with Bulldozer, so these two processor lines split that divide. That is to say that Interlagos (my old CPU) has hardware AES acceleration while Magny Cours (my new CPUs) do not and must instead perform that task in software.

As a result, encrypted FTP performance using FileZilla Server has dropped from an average of around 2 Gbps down to around 250 Mbps. The system has 24 cores available, but FileZilla is only maxing a single thread for any individual transfer. The system could provide 5 or 6 Gbps of AES throughput if the AES implementation used by FileZilla Server were multithreaded.

[boco]

Did you already have a play with the "number of threads" setting in Settings - General?

[Particle]

I did try changing that from its default of 2 to a new value of 24. I found that it doesn't appear to impact the performance (or maximum CPU utilization) of a single transfer. I suspect it would handle lots of concurrent transfers better though.

[boco]

Well, we need to wait for the developer, then.

Unfortunately, FileZilla Server is mostly legacy code and due for a rewrite (FileZilla Client has already been rewritten with version 3). So, all we currently have is that setting and the possibility to tune the TCP/IP parameters (in the OS) and buffer settings (on FZ Server's Miscellaneous settings page). Setting the buffers higher might speed up transfers.

[botg]

Unfortunately no, nothing that can be done here due to lack of library support.



I'm curious, why are you using a multi-CPU system running Windows as a storage server?

[Particle]

That is unfortunate. Perhaps if GnuTLS implements that in the future, things might be better for people in my situation.

To answer your direct question: The server started life doing just SMB file sharing in an Active Directory environment. Over time it has been asked to do other duties like light virtualization. Here recently, I've been using it to re-encode about 6 TiB of production footage stored on it into H.265 archival format. The process is exceedingly slow, so I threw another socket and more cores at it. Server hardware a few generations old remains quite capable and is fortunately also very inexpensive due to lack of demand. Energy pending, it will one day be converted to a Debian NFS file server instead. Until that happens, it's FTP to the rescue!

[boco]

FileZilla Server still uses OpenSSL.

[botg]

Sadly. FZS needs to be rewritten to use a proper TLS library.