FileZilla 3 development diary

Moderator: Project members

Message
Author
simzy39
500 Command not understood
Posts: 3
Joined: 2008-11-16 03:36
First name: heath
Last name: sims

Re: FileZilla 3 development diary

#886 Post by simzy39 » 2009-06-21 06:18

The cURL library wouldn't be good to use because it is missing alot.
Also the FTP support in cURL is not as good as FileZillas.

cURL is missing the following in FTP

4.1 PRET
4.2 Alter passive/active on failure and retry
4.3 Earlier bad letter detection
4.4 REST for large files
4.5 FTP proxy support
4.6 PORT port range
4.7 ASCII support

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#887 Post by botg » 2009-06-21 08:13

PRET is not part of any FTP standard.

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#888 Post by botg » 2009-07-01 22:14

Curse you Murphy! Your law is too cruel.

I'm building 3.2.6.1 as of writing this post, a bugfix release that's made necessary due to several independent regressions that managed to slip into 3.2.6 caused by external dependencies.
During testing, 3.2.6-rc1 worked fine, no problems spotted. But just an hour after 3.2.6 final was out, the first problem reports came in. Basically there were two problems:
  • FTPS uploads failing
  • A crash during drag & drop on OS X
The FTPS problems were caused by a regression in GnuTLS 2.8 which I did upgrade to in 3.2.6-rc1. The problem was that due to a change in GnuTLS the push function, responsible for passing encrypted data from GnuTLS to the network socket, failed to properly cope with EAGAIN errors. EAGAIN isn't a fatal error, it just means that the socket buffers are full and that the program should just try again later. The bug in GnuTLS simply caused outstanding data to be discarded. Since I'm following the GnuTLS development mailinglist I quickly found similar bug report in Pidgin. Following a hint by Simon Josefsson (one of the GnuTLS developers) I've been able to identify and work around the problem. See http://lists.gnu.org/archive/html/gnutl ... 00022.html, continued at http://lists.gnu.org/archive/html/gnutl ... 00000.html, for the bug discussion and the solution.

The bug didn't manifest itself during testing because my network is too fast, I did test FTPS in FZ 3.2.6-rc1 with GnuTLS 2.8.1 on a gigabit LAN, uploading from a slow laptop disk. EAGAIN simply didn't happen.

After the fateful release and learning about the problem, reproducing the problem was trivial. It was sufficient to set an upload speed limit in FileZilla, guaranteed transfer failures on FTPS. While trying to narrow the problem down I've noticed a similar issue happening during the TLS handshake. Oddly enough this issue also existed in earlier versions of GnuTLS such as 2.6.4 used by older versions of FileZilla. This issue too was caused by not handling EAGAIN properly. Both this as well as the previous issue had the same symptoms. But as soon as I figured out they were actually two different issues, fixing this problem as well wasn't hard. See http://lists.gnu.org/archive/html/gnutl ... 00052.html, continued at http://lists.gnu.org/archive/html/gnutl ... 00002.html, for my patch.


The crash problems on OS X were caused by a regression in wxWidgets. After hearing about the problem I first doubted that the bug reporter was using the latest version of FileZilla, since the crash report looked exactly like the ones I've seen many times in the past. This issue had originally been fixed in wxWidgets quite a while ago, but got accidentally reverted. Ironically the regression was introduced to fix another bug. Right now there are two options on OS X: Either crash or alternatively have programs that look bad. See http://groups.google.com/group/wx-dev/b ... dd39239d48# for details. I wish I could be of more help, but unfortunately I don't understand the code in question, native OS X development is too alien for me. For now I've just reverted the changes that caused the regression in 3.2.6.1

jdratlif
226 Transfer OK
Posts: 392
Joined: 2008-12-30 10:30
First name: John
Last name: Ratliff
Location: In a small white padded room.

Re: FileZilla 3 development diary

#889 Post by jdratlif » 2009-07-02 03:03

Which versions of wxWidgets for Mac are affected? 2.8.10, 2.8.9, or is it only on svn?
http://jdrrant.blogspot.com/ - CODEpendent Blog

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#890 Post by botg » 2009-07-02 07:21

Most recent stable works fine, SVN branch head is affected.

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#891 Post by botg » 2009-07-29 18:42

This is Bob. Bob is a happy and productive office worker at Initech. Let us observe how Bob uses the FileZilla FTP client. Today Bob needs to transfer some files urgently needed by Initrode, another well known company. Bob manages to log into the server and starts the transfer. Patiently Bob watches the progress bar, waiting for the transfer to complete. So far Bob is doing everything the right way. But the files are large and Bob is hungry. Since it is noon, Bob picks up jacket and leaves for lunch. He knows, when he'll come back the transfers will have finished.

Skip forward an hour. Bob has eaten and happily returns to Initech. Much to Bob's surprise he finds his boss waiting in Bob's office. His boss is very angry. The files did not transfer Bob's boss says. Initrode is going to cancel the contract with Initech Bob's boss says. Bob looks at his computer. It appears powered down. By the end of the day Bob has lost his job. He can no longer pay his rent and gets thrown out of his home. All of this only because Bob has neglected his duty and left his workplace. If you want to keep your job, don't be like Bob.


I'm sure this sounds very familiar to some of you. What happened is simple, the computer went into idle sleep even though a transfer was running. Rejoice, FileZilla will prevent that in the next version.

During transfers and other lengthy remote operations FileZilla will now tell the operating system that it cannot go to idle sleep. This feature was fairly easy to implement:
  • Under Windows, the SetThreadExecutionState function can be used for this. Really trivial to use.
  • Under OS X things are a tiny bit more complex, but still trivial. Here the IOPMAssertionCreate function can be used to tell the system not to go into idle sleep.
  • Implementing this feature under the remaining Unix(-like) systems was a lot more difficult. Perhaps the best way (and only way as far as I know) to implement it is through the org.freedesktop.PowerManagement D-Bus API. Luckily FileZilla already uses D-Bus to talk to Gnome's session manager, so actually this part too became very easy to implement.
Now while implementing this was easy, figuring out how to do it was not. No matter my search terms, Google only spew out random garbage (How I hate those utterly useless expert-sexchange and similar sites polluting search engine results). I had to navigate through a maze of haystacks trying to find the needle.

I should mention that this feature only prevents idle sleep. If your laptop runs out of battery or if you tell your computer to sleep it will still do it, FileZilla will not prevent sleep in those cases. Only during transfers and while your computer is otherwise idle it will no longer go to sleep.

User avatar
boco
Contributor
Posts: 26910
Joined: 2006-05-01 03:28
Location: Germany

Re: FileZilla 3 development diary

#892 Post by boco » 2009-07-29 21:48

There was already a discussion in JKDefrag forums about this Windows function (SetThreadExecutionState). The author (J. C. Kessels) wrote: "I used SetThreadExecutionState() in JkDefrag, but it has some problems and does not work in all cases."
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#893 Post by botg » 2009-07-29 22:44

There was already a discussion in JKDefrag forums about this Windows function (SetThreadExecutionState). The author (J. C. Kessels) wrote: "I used SetThreadExecutionState() in JkDefrag, but it has some problems and does not work in all cases."
Too bad he's become mental. Switching to proprietary junkware...

redleg
226 Transfer OK
Posts: 267
Joined: 2008-11-24 17:23

Re: FileZilla 3 development diary

#894 Post by redleg » 2009-08-01 18:43

botg wrote:This is Bob. Bob is a happy and productive office worker at Initech. Let us observe how Bob uses the FileZilla FTP client. Today Bob needs to transfer some files urgently needed by Initrode, another well known company. Bob manages to log into the server and starts the transfer.
any chance the office space (at "Initrode") running the server will get the same feature added? 8)
Network Config Guide (setup & connection issues)
FileZilla wiki (FAQs, screenshots, & "got logs?" info:)
FTP server test (plain), FTP(E)S server test (encrypted)
FTP commands (see also List_of_FTP_server_return_codes)

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#895 Post by botg » 2009-08-01 18:58

Naturally ;)

redleg
226 Transfer OK
Posts: 267
Joined: 2008-11-24 17:23

Re: FileZilla 3 development diary

#896 Post by redleg » 2009-08-01 19:04

cool... (proof FZs is still developing) :D
Network Config Guide (setup & connection issues)
FileZilla wiki (FAQs, screenshots, & "got logs?" info:)
FTP server test (plain), FTP(E)S server test (encrypted)
FTP commands (see also List_of_FTP_server_return_codes)

Swords
500 Syntax error
Posts: 14
Joined: 2006-09-21 21:36

Re: FileZilla 3 development diary

#897 Post by Swords » 2009-08-15 10:51

botg wrote:Curse you Murphy! Your law is too cruel.
that quote is superb, from where you get it?
<Signature modified due to violation of forum rules>

I like filezilla, and it is the best FTP solution I know

eddan
226 Transfer OK
Posts: 423
Joined: 2004-02-25 08:44
Location: Norway

Re: FileZilla 3 development diary

#898 Post by eddan » 2009-08-15 11:07

Swords wrote:
botg wrote:Curse you Murphy! Your law is too cruel.
that quote is superb, from where you get it?
Google results 1 - 1 of 1 for "Curse you Murphy" "Your law is too cruel". (0.07 seconds)

FileZilla Forums • View topic - FileZilla 3 development diary
12 posts - 6 authors - Last post: 1 Aug
Curse you Murphy! Your law is too cruel. I'm building 3.2.6.1 as of writing this post, a bugfix release that's made necessary due to several ...
forum.filezilla-project.org/viewtopic.php?f=3&t=64... - Cached - Similar

http://www.google.no/search?hl=en&clien ... tnG=Search
For help and support, check out the support page on the wiki.

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#899 Post by botg » 2009-08-15 15:18

Swords wrote:
botg wrote:Curse you Murphy! Your law is too cruel.
that quote is superb, from where you get it?
Made it up.

User avatar
botg
Site Admin
Posts: 35507
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: FileZilla 3 development diary

#900 Post by botg » 2009-08-18 21:32

Time for another entry in my old-fashioned diary. Diary because I don't like the term blog. As much as I'm a technology affine person I am also very skeptic about everything new. I never follow the latest fads. While others jump onto the nearest bandwagon or shiniest of failtrains, I sit back and observe. More often than not new trends turn out to be utterly useless or just old wine in new skins. I'm a slow adopter, but the stuff I do adopt completely replaces everything else before it. To give another recent example, the latest and greatest nowadays appears to be the Cloud. I'm still waiting for rain.


Now back to FileZilla, some of you might have noticed that a lot of time has passed between 3.2.7-rc1 and the final 3.2.7 release. Once again the bread jammed the floor, Murphy's law at its best. I was waiting on the GnuTLS 2.8.2 release to make it a dependency for 3.2.7 due to several bugs fixed in GnuTLS 2.8.2 which affected FileZilla. I previously talked about it in http://forum.filezilla-project.org/view ... 325#p46325. So far so good, 2.8.2 came out in time, but unfortunately it turned out to be defective, some fixes were forgotten to be merged between branches. So I had to wait a few more days on GnuTLS 2.8.3, but then I immediately released FileZilla 3.2.7 and all was well.


One of the new features in 3.2.7 is the hardened update functionality. This new update method ensures that you'll always get a legit, unmodified copy of FileZilla, provided that your original copy hadn't been altered.

FileZilla now checks for updates over a secure HTTPS connection. To prevent attacks a special certificate authority (CA) certificate has been embedded into the source of FileZilla. The certificate used by the update server needs to be signed by that CA cert or FileZilla will abort the update process and will report that something odd is going on.

In addition to the URL for the update file, the update server also transmits a SHA-512 checksum/hash. FileZilla then downloads the update file and compares the file hashes. If the hashes match the file has been downloaded successfully, and at least on Windows, the installer gets started. If the hashes fail however, the downloaded file gets deleted immediately and an error message is printed.

Post Reply