Problem with corrupted images

Need help with FileZilla Client? Something does not work as expected? In this forum you may find an answer.

Moderator: Project members

Post Reply
Message
Author
bc2005
500 Command not understood
Posts: 3
Joined: 2010-05-21 09:02
First name: Hans
Last name: Vleugels

Problem with corrupted images

#1 Post by bc2005 » 2010-05-21 09:21

I am running a 3.0.7 PL1 phpbb forum. A few weeks ago I had some problems with installing a new mod, and I had to restore the forum by a backup I made with Filezilla. (I take a backup every week, but it was the first time I ever had to restore a backup) Now it seems some images are missing or corrupted. See attachment how they look.
Screen Snaper Image.jpg
Screen Snaper Image.jpg (98.45 KiB) Viewed 13180 times
I still can find the links to these images in my SQL database tables, and they seem OK. But the images are now corrupted or no longer viewable for some reason. After some googling I read that Filezilla could be responsable for corrupting these images. They weren't downloaded binary or something. But I had selected 'Automatic' for data transfer by Filezilla? I also tried to restore older backups, but this doesn't help. I tried to upload the images with another FTP program without succes. I even tried to fix these images with software like Pixrecovery or PhotoRescue, but alas... Are my images lost now? Any info about how to restore my lost or corrupted images would be very welcome!

Another problem I noticed is that Filezilla only downloads a maximum of 9998 files per map? Very annoying if you have maps with more than 12000 files. Can this limit be changed somewhere?

Regards,
Hans

User avatar
boco
Contributor
Posts: 24359
Joined: 2006-05-01 03:28
Location: Germany

Re: Problem with corrupted images

#2 Post by boco » 2010-05-21 11:59

Are my images lost now?
If they somehow have been transferred as ASCII type, then they are probably destroyed. ASCII is only for text files because it will perform line ending conversions if necessary as described in Data Type.
Another problem I noticed is that Filezilla only downloads a maximum of 9998 files per map? Very annoying if you have maps with more than 12000 files. Can this limit be changed somewhere?
FileZilla has no limit whatsoever, it has been tested with directories containing millions of files.
The limit you encounter is on the server. Some hosters intentionally limit (read: cripple) their servers. In your case the limit is 10000 - 9998 files and the two special directories "." and "..".
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
### END SIGNATURE BLOCK ###

bc2005
500 Command not understood
Posts: 3
Joined: 2010-05-21 09:02
First name: Hans
Last name: Vleugels

Re: Problem with corrupted images

#3 Post by bc2005 » 2010-05-21 12:49

boco wrote:If they somehow have been transferred as ASCII type, then they are probably destroyed. ASCII is only for text files because it will perform line ending conversions if necessary as described in Data Type.
OK, but how come they were transferred in ASCII type when Auto was always selected as option? Could it be because of the filename with no extension?
boco wrote:FileZilla has no limit whatsoever, it has been tested with directories containing millions of files. The limit you encounter is on the server. Some hosters intentionally limit (read: cripple) their servers. In your case the limit is 10000 - 9998 files and the two special directories "." and "..".
OK, I will check this with my host...

Thx for your help!

User avatar
boco
Contributor
Posts: 24359
Joined: 2006-05-01 03:28
Location: Germany

Re: Problem with corrupted images

#4 Post by boco » 2010-05-21 13:00

OK, but how come they were transferred in ASCII type when Auto was always selected as option? Could it be because of the filename with no extension?
Yes. Filenames without extensions are treated as ASCII files by default. I already said this is a dangerous default setting for that option. Open FileZilla settings, navigate to Transfers - File types, and disable "Treat files without extension as ASCII file".

@botg: That option should be OFF by default. While it may only cause a little inconvenience for some users when it is OFF instead of ON, it irreparably destroys binary files without extensions when it is ON instead of OFF.
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
### END SIGNATURE BLOCK ###

bc2005
500 Command not understood
Posts: 3
Joined: 2010-05-21 09:02
First name: Hans
Last name: Vleugels

Re: Problem with corrupted images

#5 Post by bc2005 » 2010-05-21 13:56

OKay, now I know what happened to my files. Too bad I lost them, but maybe this can be a warning for other users so they don't get the same problem...

fdg123
500 Command not understood
Posts: 2
Joined: 2010-08-27 02:01
First name: frank
Last name: garrett

Re: Problem with corrupted images

#6 Post by fdg123 » 2010-08-27 02:14

I just had this EXACT same problem with Filezilla and Phpbb3. ALL of my pictures are gone also, 6 months of hard work in the trash. :cry: I am going to change my settings now. (and YES! you should change the default on new releases.)
BUT why are the binary files PERMENENTLY destroyed? What was done to them was an automated conversion, WHY can't they be converted back? Please explain. :?

FRANK (fdg123)

User avatar
boco
Contributor
Posts: 24359
Joined: 2006-05-01 03:28
Location: Germany

Re: Problem with corrupted images

#7 Post by boco » 2010-08-27 02:46

BUT why are the binary files PERMENENTLY destroyed?
Text files have a well defined structure:

1. Line of normal text containing printable characters only;
2. Line ending, containing CR+LF (Windows), LF (*NIX, newer Mac), CR (older Mac);
3. Repeat the above until EOF (End Of File)

The conversion simply changes line endings from one version to another.
Example: Uploading a text file from Win86 to *NIX removes all CR characters from the line endings. Downloading the same file back adds the CRs again.
This conversion only works because in text files, the only non-printable characters are the line endings $0D and $0A (and sometimes the EOF character $1A). That makes Word docs, RTF and PDF binary files, they aren't text files!

But image files contain a wild mixture of any available chars, including the line ending characters. Now imagine what happens if all CR characters are removed. File corrupt. And it is destroyed permanently, because converting back would add CRs in places where previously weren't any.
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
### END SIGNATURE BLOCK ###

fdg123
500 Command not understood
Posts: 2
Joined: 2010-08-27 02:01
First name: frank
Last name: garrett

Re: Problem with corrupted images

#8 Post by fdg123 » 2010-08-28 02:21

Ok, now I understand. :cry: but thanks for explaining it to me.
FRANK
Experiance is a hard teacher. :(

petertheartist
500 Command not understood
Posts: 1
Joined: 2013-04-21 03:07
First name: peter
Last name: warner

Re: Problem with corrupted images

#9 Post by petertheartist » 2013-04-21 03:13

Saved my bacon... After update, my preferences went back to all ASCII, didn't realize I had to change them again. It's my wife's site I was working on too

User avatar
boco
Contributor
Posts: 24359
Joined: 2006-05-01 03:28
Location: Germany

Re: Problem with corrupted images

#10 Post by boco » 2013-04-21 08:59

Preferences/settings shouldn't change on their own. Did all settings get reset?
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
### END SIGNATURE BLOCK ###

User avatar
audiopro
226 Transfer OK
Posts: 283
Joined: 2013-03-23 12:55
First name: Keith
Location: Morecambe, England
Contact:

Re: Problem with corrupted images

#11 Post by audiopro » 2013-04-21 12:09

fdg123 wrote:I just had this EXACT same problem with Filezilla and Phpbb3. ALL of my pictures are gone also, 6 months of hard work in the trash. :cry:
I don't understand how this happened to you.
You uploaded the files from your local computer to the server - found they were all corrupted because you transferred them in ascii mode by mistake.

Why didn't you just upload them again but this time as binaries?

As for 6 months work in the trash - backups are essential for all important files.
Morecambe - Where the sun goes at night

rinu
500 Command not understood
Posts: 1
Joined: 2014-03-26 19:32
First name: Rauno
Last name: M

Re: Problem with corrupted images

#12 Post by rinu » 2014-03-26 19:45

I've created an account here just to say how I fixed my images. This is the first real Google search result so I'd like the next person to end up here to see the solution.

In my case the files corrupted when transferring from a Linux system to Windows.
As I learned from this topic that automagically replaced all \n with \r\n.
What I did was open all image files in the files directory with Notepad++.
Then used the replace function (Ctrl+H) to replace all \r\n in all files with \n. Search mode setting need to be set to Extended.
Save all files and that's it - images look great again.

I also changed the FileZilla setting not to treat files without extensions as ASCII to avoid this issue in the future.

User avatar
boco
Contributor
Posts: 24359
Joined: 2006-05-01 03:28
Location: Germany

Re: Problem with corrupted images

#13 Post by boco » 2014-03-26 22:11

If that worked you were really lucky. The line ending conversion replaced \n with \r\n - but what about places where there was already \r\n before? I repeat: You were very lucky because your images didn't seem to have such spots.

Of course you can always attempt to do the reverse operation, but you need to be aware that it is far from being reliable. Smaller images = Higher chance.
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
### END SIGNATURE BLOCK ###

Post Reply