Multithreaded Encryption for FTP Over TLS
Moderator: Project members
Multithreaded Encryption for FTP Over TLS
Are there any options for multithreaded software encryption that can be enabled in FileZilla server? If not, I'd like to request it.
I recently upgraded a storage server of mine from a single Opteron 6234 to dual Opteron 6180 SE processors. The former is an Interlagos based processor (ie Bulldozer). The latter are Magny Cours based processors (ie Thuban). AMD introduced hardware AES acceleration in their chips starting with Bulldozer, so these two processor lines split that divide. That is to say that Interlagos (my old CPU) has hardware AES acceleration while Magny Cours (my new CPUs) do not and must instead perform that task in software.
As a result, encrypted FTP performance using FileZilla Server has dropped from an average of around 2 Gbps down to around 250 Mbps. The system has 24 cores available, but FileZilla is only maxing a single thread for any individual transfer. The system could provide 5 or 6 Gbps of AES throughput if the AES implementation used by FileZilla Server were multithreaded.
I recently upgraded a storage server of mine from a single Opteron 6234 to dual Opteron 6180 SE processors. The former is an Interlagos based processor (ie Bulldozer). The latter are Magny Cours based processors (ie Thuban). AMD introduced hardware AES acceleration in their chips starting with Bulldozer, so these two processor lines split that divide. That is to say that Interlagos (my old CPU) has hardware AES acceleration while Magny Cours (my new CPUs) do not and must instead perform that task in software.
As a result, encrypted FTP performance using FileZilla Server has dropped from an average of around 2 Gbps down to around 250 Mbps. The system has 24 cores available, but FileZilla is only maxing a single thread for any individual transfer. The system could provide 5 or 6 Gbps of AES throughput if the AES implementation used by FileZilla Server were multithreaded.
Re: Multithreaded Encryption for FTP Over TLS
Did you already have a play with the "number of threads" setting in Settings - General?
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
Re: Multithreaded Encryption for FTP Over TLS
I did try changing that from its default of 2 to a new value of 24. I found that it doesn't appear to impact the performance (or maximum CPU utilization) of a single transfer. I suspect it would handle lots of concurrent transfers better though.
Re: Multithreaded Encryption for FTP Over TLS
Well, we need to wait for the developer, then.
Unfortunately, FileZilla Server is mostly legacy code and due for a rewrite (FileZilla Client has already been rewritten with version 3). So, all we currently have is that setting and the possibility to tune the TCP/IP parameters (in the OS) and buffer settings (on FZ Server's Miscellaneous settings page). Setting the buffers higher might speed up transfers.
Unfortunately, FileZilla Server is mostly legacy code and due for a rewrite (FileZilla Client has already been rewritten with version 3). So, all we currently have is that setting and the possibility to tune the TCP/IP parameters (in the OS) and buffer settings (on FZ Server's Miscellaneous settings page). Setting the buffers higher might speed up transfers.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
Re: Multithreaded Encryption for FTP Over TLS
Unfortunately no, nothing that can be done here due to lack of library support.
I'm curious, why are you using a multi-CPU system running Windows as a storage server?
I'm curious, why are you using a multi-CPU system running Windows as a storage server?
Re: Multithreaded Encryption for FTP Over TLS
That is unfortunate. Perhaps if GnuTLS implements that in the future, things might be better for people in my situation.
To answer your direct question: The server started life doing just SMB file sharing in an Active Directory environment. Over time it has been asked to do other duties like light virtualization. Here recently, I've been using it to re-encode about 6 TiB of production footage stored on it into H.265 archival format. The process is exceedingly slow, so I threw another socket and more cores at it. Server hardware a few generations old remains quite capable and is fortunately also very inexpensive due to lack of demand. Energy pending, it will one day be converted to a Debian NFS file server instead. Until that happens, it's FTP to the rescue!
To answer your direct question: The server started life doing just SMB file sharing in an Active Directory environment. Over time it has been asked to do other duties like light virtualization. Here recently, I've been using it to re-encode about 6 TiB of production footage stored on it into H.265 archival format. The process is exceedingly slow, so I threw another socket and more cores at it. Server hardware a few generations old remains quite capable and is fortunately also very inexpensive due to lack of demand. Energy pending, it will one day be converted to a Debian NFS file server instead. Until that happens, it's FTP to the rescue!
Re: Multithreaded Encryption for FTP Over TLS
FileZilla Server still uses OpenSSL.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
Re: Multithreaded Encryption for FTP Over TLS
Sadly. FZS needs to be rewritten to use a proper TLS library.