character encoding issues

[thwaller]

This happens in both Filezilla 3.46.3 (from my package manager) and 3.5 from your site. Using Ubuntu 18.04.5.

I upload my files to the server using Filezilla and they all upload properly, ex the file "Sautéed Cabbage and Kale.html" shows on the server properly after upload. In FileZilla however, it appears as "Sautéed Cabbage and Kale.html" after the upload. I checked this using CrossFTP, and the issue is not there. The files all appear properly.

I noticed this as a problem when I tried to delete files with such encoding issues in FileZilla and they could not be deleted. I needed to go to the server itself and delete them.

Why does FileZilla upload properly, but display/read them improperly after?

[boco]

This should not happen if the FTP server properly supports UTF-8. If the FTP server does NOT support UTF-8, any characters outside of English letters, numbers, and very few other characters (namely 7-bit US-ASCII) are not supported. That's a limitation of the original FTP protocol, and UTF-8 is the only supported way to overcome it.

Other clients purposely violate the FTP standards by using custom encoding. While you can force custom encoding in FileZilla, too (see Site Manager), we don't provide any support for it.

[thwaller]

Could you kindly clarify "properly supports UTF-8"? On the server itself, the file manager shows all in proper format and on the web when I browse the directory it is also in proper format. Uploading in FileZilla is fine, but viewing/editing/deleting is the problem.

[boco]

We do not speak about anything other than the FTP server software. All other services, like direct or HTTP access to the files, are probably fully UTF-8 or Unicode capable. But FTP is old, very old. When it was born, the only language used was English. Thus, in its basic form, FTP does not understand anything other than English. Plus, it does not support negotiating other encoding schemes (also known as character sets).

International characters are supported by an extension to the FTP protocol, named UTF-8. Both FTP client and FTP server must support it for proper operation.

If you get the following line in the upper log pane (right click and select "Show detailed log" before connect) upon connection, the server does not support UTF-8.

Code: Select all

06:51:53	Status:	Server does not support non-ASCII characters.

FileZilla does not actively prevent using accented letters like é in filenames and will send it to the server as-is, using the codepage set in your system. The outcome is completely unknown, it may work or not.

Now to the file listing. The server will send the listing using its own codepage. Obviously, your system has a different codepage, and thus FileZilla doesn't fully understand it. The result is garbled text. I don't know the other client you mentioned, but it probably tried to guess the server codepage and was lucky.


Unfortunately, there is no easy solution. Possible options are:
1. Convince the FTP server administrator to update to a software supporting UTF-8. These days, there is NO reason not to use it.
2. Use only English in your paths and filenames. English is understood by every proper FTP server out there.
3. Try to guess the servers codepage and enter it as custom charset in the Site Manager.

[thwaller]

Thank you kindly for your detailed response.

I found that if I set to force UTF-8, all works fine and as expected. I understand that this is not a proper solution, but it solves all of the issues.

I also understand your comment on using English and it is a valid solution. I am not a Windows user, so I am not so used to being restricted on my characters I can use in file names. As a firm believer in UTF-8 as a master encoding, I have no intentions to change for any server. So, I am happy I can get around this by forcing UTF-8 in the FileZilla settings.

Thanks again for the details. Much appreciated and my query is solved.

[boco]

Well, if it works when forcing UTF-8, then the server DOES support it, but it fails to announce it when being asked. If you do not experience any side-effects, forcing UTF-8 is the right solution for this server.

Note that forcing UTF-8 on a server not supporting it would not work, as you cannot force anything that isn't there. So, you're actually lucky.