FileZilla does not like leading space in directory names...

[electroglyph]

Connection type: SFTP
FileZilla version: 3.7.0.2
Error: "no such file or directory"

FileZilla wasn't letting me navigate to a certain directory. I busted out Putty and SSHed into the root directory. It was there that I noticed the directory name actually has a leading space.

FileZilla log:

Code: Select all

06-05 02:58:25 5464 9 Status: Connecting to <snip>...
2013-06-05 02:58:25 5464 9 Trace: Going to execute "C:\Program Files (x86)\FileZilla FTP Client\fzsftp.exe"
2013-06-05 02:58:25 5464 9 Response: fzSftp started
2013-06-05 02:58:25 5464 9 Trace: CSftpControlSocket::ConnectParseResponse(fzSftp started)
2013-06-05 02:58:25 5464 9 Trace: CSftpControlSocket::SendNextCommand()
2013-06-05 02:58:25 5464 9 Trace: CSftpControlSocket::ConnectSend()
2013-06-05 02:58:25 5464 9 Command: open <snip> 22
2013-06-05 02:58:25 5464 9 Trace: Looking up host <snip>
2013-06-05 02:58:25 5464 9 Trace: Connecting to <snip> port 22
2013-06-05 02:58:25 5464 9 Trace: Server version: SSH-2.0-OpenSSH_5.5p1 Debian-6+squeeze3
2013-06-05 02:58:25 5464 9 Trace: Using SSH protocol version 2
2013-06-05 02:58:25 5464 9 Trace: We claim version: SSH-2.0-PuTTY_Local:_May_26_2013_17:53:09
2013-06-05 02:58:25 5464 9 Trace: Doing Diffie-Hellman group exchange
2013-06-05 02:58:25 5464 9 Trace: Doing Diffie-Hellman key exchange with hash SHA-256
2013-06-05 02:58:26 5464 9 Trace: Host key fingerprint is:
2013-06-05 02:58:26 5464 9 Trace: ssh-rsa 2048 <snip>
2013-06-05 02:58:26 5464 9 Trace: Initialised AES-256 SDCTR client->server encryption
2013-06-05 02:58:26 5464 9 Trace: Initialised HMAC-SHA1 client->server MAC algorithm
2013-06-05 02:58:26 5464 9 Trace: Initialised AES-256 SDCTR server->client encryption
2013-06-05 02:58:26 5464 9 Trace: Initialised HMAC-SHA1 server->client MAC algorithm
2013-06-05 02:58:26 5464 9 Command: Pass: **********
2013-06-05 02:58:26 5464 9 Trace: Sent password
2013-06-05 02:58:26 5464 9 Trace: Access granted
2013-06-05 02:58:26 5464 9 Trace: Opened channel for session
2013-06-05 02:58:27 5464 9 Trace: Started a shell/command
2013-06-05 02:58:27 5464 9 Status: Connected to <snip>
2013-06-05 02:58:27 5464 9 Trace: CSftpControlSocket::ConnectParseResponse()
2013-06-05 02:58:27 5464 9 Trace: CSftpControlSocket::ResetOperation(0)
2013-06-05 02:58:27 5464 9 Trace: CControlSocket::ResetOperation(0)
2013-06-05 02:58:27 5464 9 Trace: CFileZillaEnginePrivate::ResetOperation(0)
2013-06-05 02:58:27 5464 9 Status: Retrieving directory listing...
<snip>
2013-06-05 02:58:30 5464 9 Command: cd "downloads2"
2013-06-05 02:58:31 5464 9 Response: New directory is: "/home/torrent/downloads2"
2013-06-05 02:58:31 5464 9 Trace: CSftpControlSocket::ResetOperation(0)
2013-06-05 02:58:31 5464 9 Trace: CControlSocket::ResetOperation(0)
2013-06-05 02:58:31 5464 9 Trace: CSftpControlSocket::ParseSubcommandResult(0)
2013-06-05 02:58:31 5464 9 Trace: CSftpControlSocket::ListSubcommandResult()
2013-06-05 02:58:31 5464 9 Trace:   state = 1
2013-06-05 02:58:31 5464 9 Trace: CSftpControlSocket::SendNextCommand()
2013-06-05 02:58:31 5464 9 Trace: CSftpControlSocket::ListSend()
2013-06-05 02:58:31 5464 9 Trace:   state = 2
2013-06-05 02:58:31 5464 9 Command: ls
2013-06-05 02:58:31 5464 9 Status: Listing directory /home/torrent/downloads2
<snip>
2013-06-05 02:58:36 5464 9 Listing: drwxrwxr-x  1695 torrent  www-data   135168 Jun  5 01:15 .
2013-06-05 02:58:36 5464 9 Listing: drwxrwxr-x    8 torrent  www-data     4096 May  5 22:17 ..
2013-06-05 02:58:36 5464 9 Listing: drwxrwxr-x    2 torrent  www-data     4096 Jan 11  2012 slidecamp - Slidecamp (flac)
2013-06-05 02:58:36 5464 9 Listing: drwxrwxr-x    2 torrent  www-data     4096 Mar 16  2012 1995 - Science Fiction - They Tried To Grab Me (FLAC Vinyl)
2013-06-05 02:58:36 5464 9 Listing: drwxrwxrwx    2 torrent  www-data     4096 Jan  6 09:25 Dimitri_Vangelis_and_Wyman_feat._Jonny_Rose-Pieces_Of_Light-PROMO-CDR2-FLAC-2012-WRE
2013-06-05 02:58:36 5464 9 Listing: drwxrwxr-x    2 torrent  www-data     4096 Jan 28  2012 Burning Oil
2013-06-05 02:58:36 5464 9 Listing: drwxrwxrwx    2 torrent  www-data     4096 Jun  5 01:14  Coil - 1999 - Musick To Play In The Dark Vol. 1
2013-06-05 02:58:36 5464 9 Listing: drwxrwxrwx    2 torrent  www-data     4096 Jan  6 09:26 Various - Thrust Retrospective Vol 1 Mixed By Lee Coombs (2012) (THRUSTCD002) WEB FLAC
2013-06-05 02:58:36 5464 9 Listing: drwxrwxrwx    2 torrent  www-data     4096 Apr 17 04:11 Kill the Noise - Roots Remixed EP (2012) [FLAC]
<snip>
2013-06-05 02:58:36 5464 9 Trace: CSftpControlSocket::ListParseResponse()
2013-06-05 02:58:36 5464 9 Trace: CSftpControlSocket::ResetOperation(0)
2013-06-05 02:58:36 5464 9 Trace: CControlSocket::ResetOperation(0)
2013-06-05 02:58:36 5464 9 Status: Directory listing successful
2013-06-05 02:58:36 5464 9 Trace: CFileZillaEnginePrivate::ResetOperation(0)
2013-06-05 02:58:47 5464 9 Status: Retrieving directory listing...
2013-06-05 02:58:47 5464 9 Trace: CSftpControlSocket::SendNextCommand()
2013-06-05 02:58:47 5464 9 Trace: CSftpControlSocket::ChangeDirSend()
2013-06-05 02:58:47 5464 9 Command: cd "Coil - 1999 - Musick To Play In The Dark Vol. 1"
2013-06-05 02:58:47 5464 9 Error: Directory /home/torrent/downloads2/Coil - 1999 - Musick To Play In The Dark Vol. 1: no such file or directory
2013-06-05 02:58:47 5464 9 Trace: CSftpControlSocket::ResetOperation(2)
2013-06-05 02:58:47 5464 9 Trace: CControlSocket::ResetOperation(2)
2013-06-05 02:58:47 5464 9 Trace: CSftpControlSocket::ParseSubcommandResult(2)
2013-06-05 02:58:47 5464 9 Trace: CSftpControlSocket::ListSubcommandResult()
2013-06-05 02:58:47 5464 9 Trace:   state = 1
2013-06-05 02:58:47 5464 9 Trace: CSftpControlSocket::ResetOperation(2)
2013-06-05 02:58:47 5464 9 Trace: CControlSocket::ResetOperation(2)
2013-06-05 02:58:47 5464 9 Error: Failed to retrieve directory listing
2013-06-05 02:58:47 5464 9 Trace: CFileZillaEnginePrivate::ResetOperation(2)

Putty log:

Code: Select all

torrent@www:~/downloads2$ cd " Coil - 1999 - Musick To Play In The Dark Vol. 1"
torrent@www:~/downloads2/ Coil - 1999 - Musick To Play In The Dark Vol. 1$

I'm not sure how tolerant your parser needs to be to retain maximum compatibility....but perhaps if your parser could use the "." and ".." listing to 'calibrate' itself to detect if a single space is the delimiter for date and directory name?

[audiopro]

Post subject: FileZilla does not like leading space in directory names...

It is not only Filezilla that doesn't like leading / trailing spaces in directory names, variable names and file names. They are a bad idea in any situation where the name may need to be copied or typed.

[electroglyph]

Post subject: FileZilla does not like leading space in directory names...

It is not only Filezilla that doesn't like leading / trailing spaces in directory names, variable names and file names. They are a bad idea in any situation where the name may need to be copied or typed.

Just because other software (source?) doesn't like leading spaces doesn't make it okay for FileZilla to have broken support, too.

The problem here is that I didn't create the directory. I didn't know it had a leading space, and neither did FileZilla. It was only after I SSHed into my server that I found the discrepancy. Linux software (Apache, PHP, rtorrent) didn't have a problem with the directory name. Just tested on Windows too and having a leading space wasn't a problem for the programs I tested.

I agree it's a bad idea, but that doesn't mean it shouldn't be supported.

[electroglyph]

This fixes it, not sure if it breaks support for any particular servers though. It just sets include_whitespace to TRUE for the filename token in the unix parser.

Since the filename is the last thing to be parsed I think it's a fairly safe change to make???

Code: Select all

--- C:\dev\msys\home\ANON\source\filezilla-3.7.0.2-orig\src\engine/directorylistingparser.cpp	2013-04-04 20:30:06.000000000 -0700
+++ C:\dev\msys\home\ANON\source\filezilla-3.7.0.2\src\engine/directorylistingparser.cpp	2013-06-07 21:06:36.001183100 -0700
@@ -995,7 +995,8 @@
 		}
 
 		// Get the filename
-		if (!pLine->GetToken(++index, token, 1))
+		// if (!pLine->GetToken(++index, token, 1))
+		if (!pLine->GetToken(++index, token, 1, 1))
 			continue;
 
 		entry.name = token.GetString();

[botg]

Thanks, though unfortunately it won't work. There are servers which pad the various fields in the listing with additional spaces:

Code: Select all

-rw-r--r--  1 user     group     4096    Apr     8     5:12      foo
-rw-r--r--  1 blehblah group    94375    May    17    23:12      bar

[audiopro]

I don't know the setup there ie. what is accessing the directory via a program etc. but the obvious solution would be to rename the directory to save possible problems in the future.

[boco]

LIST and SFTP listing output isn't standardized. That's the whole issue here. FTP servers with MLSx support should work fine.

[electroglyph]

Thanks, though unfortunately it won't work. There are servers which pad the various fields in the listing with additional spaces:

Code: Select all

-rw-r--r--  1 user     group     4096    Apr     8     5:12      foo
-rw-r--r--  1 blehblah group    94375    May    17    23:12      bar

I was afraid of that.

So basically the only way to do it is to make a complicated parser even more complicated... that sucks

Sorry if I should know this, but is there currently a way to test the directory parser against known server configurations? If not, do you have examples of all the unix directory listing formats? Obviously this isn't an incredibly high priority, but it might be fun for me to look at later. I've only written one actual C++ program, but in my defense it was a self-injecting DLL with lots of naked ASM functions with custom written hooks, so it was at least slightly more advanced than a hello world.

I don't know the setup there ie. what is accessing the directory via a program etc. but the obvious solution would be to rename the directory to save possible problems in the future.

You think I could set up FileZilla's build environment, scroll through 3000+ lines of C++ code, locate a potential fix, and compile a test build all without knowing how to rename a directory? I appreciate the help, but come on man

LIST and SFTP listing output isn't standardized. That's the whole issue here. FTP servers with MLSx support should work fine.

Yeah, I figured that out, but I was hoping it was more standardized than it turned out to be. I can't have FTP on my server for security reasons. I think it's still an interesting problem to solve if it's done correctly and efficiently.

[electroglyph]

Hmm....I wonder if every host that supports "ls -l" supports "ls -lQ"?? That could help in making an alternative parser. Actually, we could use the same exact parser we're using now, and just strip the quotes off. The only thing I'm unsure of is how it would affect FileZilla's handling of symlinks

[audiopro]

You think I could set up FileZilla's build environment, scroll through 3000+ lines of C++ code, locate a potential fix, and compile a test build all without knowing how to rename a directory? I appreciate the help, but come on man

I am not saying you do not know how to rename a directory, just that you seem to be looking for a complicated solution to what is a very simple problem.

I have no doubt your knowledge of C++ is far in excess of mine but I think you are missing the point here.
You could either dedicate the rest of your life creating a workround which may or may not work, or you could simply rename the directory.

Back in the 60's, the USA spent tens of thousands of dollars and dedicated hundreds of man hours developing a pen which could be used both on earth and in a zero gravity space environment. After many failed attempts they finally managed to produce a marvellously engineered item which worked really well.
The Russians on the other hand used a pencil.

[electroglyph]

You think I could set up FileZilla's build environment, scroll through 3000+ lines of C++ code, locate a potential fix, and compile a test build all without knowing how to rename a directory? I appreciate the help, but come on man

I am not saying you do not know how to rename a directory, just that you seem to be looking for a complicated solution to what is a very simple problem.

I have no doubt your knowledge of C++ is far in excess of mine but I think you are missing the point here.
You could either dedicate the rest of your life creating a workround which may or may not work, or you could simply rename the directory.

Back in the 60's, the USA spent tens of thousands of dollars and dedicated hundreds of man hours developing a pen which could be used both on earth and in a zero gravity space environment. After many failed attempts they finally managed to produce a marvellously engineered item which worked really well.
The Russians on the other hand used a pencil.

Sometimes the fun is in the engineering, not the final product. In my case it's not as simple as just renaming a single directory. I don't know beforehand what directories are going to be named. In all honesty doing the checks on my end to make sure all files & directories with leading & trailing whitespace are sanitized would take at least as much effort (probably much more) as working on FileZilla's parser. It's also not as simple as running a cron script to rename the directories either because I have database entries and other BS to muck around with. Besides, every programmer has a desire to do things the proper way. It's possible to parse the directory listing accurately for every server, that's the proper way to solve the problem. (no disrespect to FileZilla, the parser actually looks good)

Using the --quote-name option of 'ls' is actually pretty straightforward, and if one of the experts around here agrees it will work it shouldn't be too much trouble. It's obviously not a huge deal, but looking over the bug tracker I've seen people going back years who have had issues with it, so I know at least a handful of people would benefit.

This thread is already getting absurdly long, but one more thing...FileZilla kicks ass. I tried leaving it before, when I had charset issues, and I just couldn't do it. I'm probably one of the few people out there that actually uses FileZilla to it's full potential, and it saves me a ton of time. So for me it's worth it to spend a little time to make it even better.

[botg]

Sorry if I should know this, but is there currently a way to test the directory parser against known server configurations? If not, do you have examples of all the unix directory listing formats?

Have a look at tests/dirparsertest.cpp, there are about 80 different types of directory listings supported by FileZilla and encountered in the wild. Some of the differences are very subtle.

[audiopro]

I didn't know the full circumstances, as I mentioned in an earlier post so I wasn't sure how complex the problem was for you. What is creating directory names with leading spaces?

I agree Filezilla is a fine FTP program although it is not the best I have ever used. An old program, I have, dating back to 1995 is far better as it is just an FTP program without the bells and whistles. Unfortunately, it is not compatible with some of the newer servers. They upgraded it recently and the new version is awful.

[boco]

WS_FTP #LE?

[audiopro]

Thats the one but corrupts Perl scripts on one server and times out after 20 seconds on another