FileZilla 3 development diary

Moderator: Project members

Message
Author
User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#961 Post by botg » 2013-01-20 08:21

Perhaps, if somebody submits high-quality patches.

kadushkin
504 Command not implemented
Posts: 8
Joined: 2013-01-18 12:40

Re: FileZilla 3 development diary

#962 Post by kadushkin » 2013-01-20 09:14

This means that I need to do myself?

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#963 Post by botg » 2013-01-20 10:11

No, it merely means that somebody other than myself has to do it.

kadushkin
504 Command not implemented
Posts: 8
Joined: 2013-01-18 12:40

Re: FileZilla 3 development diary

#964 Post by kadushkin » 2013-01-22 07:03

And you can pass my request to the developers?

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#965 Post by botg » 2013-01-22 10:44

I am "the developers".

kadushkin
504 Command not implemented
Posts: 8
Joined: 2013-01-18 12:40

Re: FileZilla 3 development diary

#966 Post by kadushkin » 2013-01-22 13:49

botg wrote:No, it merely means that somebody other than myself has to do it.
But who can do it?

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#967 Post by botg » 2013-01-22 14:09

Any skilled developer can do it.

kadushkin
504 Command not implemented
Posts: 8
Joined: 2013-01-18 12:40

Re: FileZilla 3 development diary

#968 Post by kadushkin » 2013-01-27 14:58

I made ​​myself, because I'm a developer too :)

Create at "C:\Program Files\FileZilla\resources\alias" file "path_alias.xml"

Code: Select all

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<FileZilla3>
    <Config>
    <Alias_count>2</Alias_count>
   </Config>

  <Path_alias1>
    <FTP_Path>ftp://my@mysite.ru/domain/my/big-e.ru</FTP_Path>
    <HTTP_Path>http://big-e.ru</HTTP_Path>
   </Path_alias1>

  <Path_alias2>
    <FTP_Path>ftp://me.nhl.com@nhl.com</FTP_Path>
    <HTTP_Path>http://egor.cznhl.com</HTTP_Path>
   </Path_alias2>
</FileZilla3>
Add this code to RemoteListView.cpp

Code: Select all

#include "xmlfunctions.h"
....
bool ReplaceAliasData(wxString& sURL, const wxString& aliasPath)
{
wxString sPath_alias=_T("");;
wxString FTP_Path;
wxString HTTP_Path;
wxString sAlias_count;

	wxFileName fn(wxGetApp().GetResourceDir() + aliasPath, _T("path_alias.xml"));
	TiXmlElement* pDocument = GetXmlFile(fn.GetFullPath());
	if (!pDocument)
		return false;


TiXmlElement* pConfig = pDocument->FirstChildElement("Config");
if (pConfig)
{
sAlias_count = GetTextElement(pConfig, "Alias_count");
}


	if (!sAlias_count)
		return false;

int Alias_count=0;
Alias_count=wxAtoi(sAlias_count);


wxString ii;
TiXmlElement* pAlias; 
for (int i = 1; i <= Alias_count; i++)
{
//ii.Format(wxT("%d"),i);
ii=wxString::Format(wxT("%i"),i);
sPath_alias=_T("Path_alias");
sPath_alias+=ii;

	pAlias = pDocument->FirstChildElement(sPath_alias.mb_str());
	if (pAlias)
	{
		FTP_Path = GetTextElement(pAlias, "FTP_Path");
		HTTP_Path = GetTextElement(pAlias, "HTTP_Path");
if ((HTTP_Path!= _T("")) && (FTP_Path!= _T(""))) {sURL.Replace(FTP_Path, HTTP_Path);}
		}
}

	delete pDocument->GetDocument();



	return pAlias != 0;
}
.....

url.Replace(_T(" "), _T("%20")); //<- this string exist
ReplaceAliasData(url, _T("alias/"));
....
urls.Replace(_T(" "), _T("%20")); //<- this string exist
ReplaceAliasData(urls, _T("alias/"));
How to use:
Press "Copy URL(s) to Clipboard" in context menu at RemoteListView field only

Thanks.

Bokhalza
500 Command not understood
Posts: 1
Joined: 2014-02-20 13:17
First name: Lou
Last name: Kay

Re: FileZilla 3 development diary

#969 Post by Bokhalza » 2014-02-20 13:21

Excellent botg!
Security is mostly a superstition. It does not exist in nature, nor do the children of men as a whole experience it.
<Third-party URLs in signatures not allowed>

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#970 Post by botg » 2014-05-02 19:52

I want to use some C++11 in FileZilla and FileZilla Server, but until recently I couldn't. Support for old operating systems is one reason, in particular Windows XP and OS X 10.4.

With FileZilla Server it was easy, only a single platform to support. I've upgraded the server's build infrastructure to use Visual Studio 2013. VS2013 doesn't support XP anymore without lots of tinkering. Did it once, it was painful and took ages. Then the OpenSSL bug happened, I dropped XP for good then.

With the client it's more complex. Easiest part is Linux, nothing to be done there, the infrastructure already supports most of C++11. Windows is harder, in particular since the official binaries are being cross-compiled on Debian using MinGW. Switching to Debian Jessie (aka testing) on the build system allowed me to switch to a far newer version of the MinGW cross-compiler. Cross-compiling the dependencies and then FileZilla using the new infrstructure worked without a glitch, I had expected far worse, MinGW used to be so problematic. Last on the agenda is OS X, the OS X binaries of FileZilla are built on an ancient Mac Mini [*] (isolated behind a non-consumer-grade firewall) running OS X 10.4. Apart from not supporting C++11, it's time to drop support for that unsupported operating system and turn over the box to a museum or something :) I've already got a new Mac Mini running OS X 10.9, now I just need to set up the infrastructure on it.

Dropping support for OS X 10.4/10.5 leaves me slightly unhappy, it's after all the only big-endian system I have access too. It's good practice to write code that can handle either endianess, in particular since code parts not working with a different endianess often have other problems as well. This is particularly relevant as endian conversion often happens during I/O serialisation. Not doing that right can lead to security vulnerabilities.

[*]A lovely beige box it is. Why is it that over time, new white hardware inevitably turns yellow over time? Black on the other hand, not only does it look better, it also stays black over time.

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#971 Post by botg » 2014-05-03 13:09

Crap. OS X 10.9 is too new to build wxWidgets 2.8 on. Currently trying to somehow install the base SDK for OS X 10.6 on it, which is the newest version supported by wx 2.8.

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#972 Post by botg » 2014-05-03 20:59

Done. I've managed to build install the SDK for OS X 10.6 on 10.9 and build binaries targetting 10.4. There's a great German expression for things like this: Von Hinten durch die Brust ins Auge.

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#973 Post by botg » 2014-05-08 17:51

The future of OS X support

The good news first: OS X in general support is there to stay.

The bad: Minimum supported OS X version will be OS X 10.7, supporting older versions will result in a lot of ugliness.

Some data:

Code: Select all

CPU   OS X  Share
x86   10.9  59.2%
x86   10.8  14.7%
x86   10.7  11.0%
x86   10.6  12.0%
ppc   10.6   0.8%
x86   10.5   1.5%
ppc   10.5   0.4%

OS X users out of all users:   11%
What does this mean?

This will mean that in the near future, roughly 15% of all OS X users will no longer receive FileZilla updates.

Apart from that, there's something interesting in the data: A small number of OS X users is emulating the PowerPC binaries on their x86 CPU, as OS X 10.5 is the latest version of OS X to ever support PPC.

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#974 Post by botg » 2014-10-07 19:58

Post-mortem of a bug in the auto-updater

On platforms without a package manager, a reliable auto-update mechanism is of immense importance to encourage the use of the latest software version. This is especially true in networked programs where security vulnerabilities can have vast consequences.
For years now FileZilla has had an auto-update mechanism. A version with a broken auto-update would mean that many, if not most users of FileZilla would remain unaware of a new version with potentially critical fixes.

As part of the release process for new FileZilla versions I always test the auto-update functionality in the new version. I did successfully test 3.9.0.3 when it got released and also tested 3.9.0.4 with success prior to its release. I was surprised to find out shortly after releasing the new version that the previous 3.9.0.3 was unable to download the update.

I quickly went to analyze the issue: Something wrong with a mirror? Something wrong in my network? Something wrong with FileZilla itself? Debug logs in combination with a packet dump created with Wireshark quickly showed that the problem was in FileZilla itself.

Knowing that the problem is with FileZilla, I then pulled the new version from the update mechanism to prevent additional harm.

From the logs and the dumps I had a rough idea during which part of the download process the bug would trigger, though I had to tediously step through the code with a debugger to actually narrow it down. Interestingly during debugging the bug would not always happen, indicating that this bug is sensitive to timing. This fact helped me a lot in finding the cause of the bug. I remembered that in 3.9.0.3 I had made a few changes to the network code to fix a rare crash on connection termination. This change introduced a regression, depending on how the connection was closed, when reusing the socket instance not all state from the previous connection was being reset. The fix for this regression was then simple: http://svn.filezilla-project.org/filezi ... threv=6062

Further analysis showed that the bug only affected HTTP redirect handling, something the primary download servers makes ample use of to spread the traffic across secondary mirrors. Incidentally the update test infrastructure didn't use HTTP redirects, which is reason why this bug slipped through unnoticed. Much additional testing on both the affected and the fixed versions did not reveal any more problems, so I pushed out 3.9.0.5.

With the bug fixed and the fixed version released, I still had to get the fix to the users of the affected users somehow as due to the bug they couldn't get the update. Fortunately the bug only affected redirects and the primary download servers are under my direct control. Since the auto-update tests in the past always were successful, I knew that there had to be some way to get 3.9.0.3 to update. Eventually I managed to change the server-side auto-update back-end, coupled with some crazy rewrite rules for Apache's mod_rewrite, to avoid using HTTP redirects, at the expense of additional traffic on the primary servers.

Conclusion

You can see the impact this workaround had on traffic consumption on this traffic graph from one of the servers:
1412707008.png
1412707008.png (10.47 KiB) Viewed 31605 times
During September, the primary download servers had to handle around roughly 12 TiB of extra data, most of it during the first week after the 3.9.0.5 release. The graph also shows what looks like an exponential decline in 3.9.0.3 usage and you can clearly see the weekends. While not shown here, on hourly traffic graphs you could even see when office hours in Europe and the Americas begin and end.

In conclusion, I hope this post has provided at least some insight into one of the many challenges creating reliable auto-updaters and how to deal with unforeseen bugs in the auto-updater.

User avatar
botg
Site Admin
Posts: 35491
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#975 Post by botg » 2015-01-21 16:20

In the next version of FileZilla the following FTP over TLS changes will be made:
  • Support for the insecure RC4 algorithm will be removed
  • The certificate verification dialog will show subject alternative names
  • When a TLS alert is received, it now says which alert got received
  • Improved error reporting on underlying socket instead of the generic "error in the push/pull function"
In some future version, the following changes will be made:
  • Mismatch of server hostname with the certificate's common name or subject alternate name will become an error.
  • Session reuse on the data connection will be mandatory

Post Reply