Problem with corrupted images
Moderator: Project members
-
- 500 Command not understood
- Posts: 3
- Joined: 2010-05-21 09:02
- First name: Hans
- Last name: Vleugels
Problem with corrupted images
I am running a 3.0.7 PL1 phpbb forum. A few weeks ago I had some problems with installing a new mod, and I had to restore the forum by a backup I made with Filezilla. (I take a backup every week, but it was the first time I ever had to restore a backup) Now it seems some images are missing or corrupted. See attachment how they look.
I still can find the links to these images in my SQL database tables, and they seem OK. But the images are now corrupted or no longer viewable for some reason. After some googling I read that Filezilla could be responsable for corrupting these images. They weren't downloaded binary or something. But I had selected 'Automatic' for data transfer by Filezilla? I also tried to restore older backups, but this doesn't help. I tried to upload the images with another FTP program without succes. I even tried to fix these images with software like Pixrecovery or PhotoRescue, but alas... Are my images lost now? Any info about how to restore my lost or corrupted images would be very welcome!
Another problem I noticed is that Filezilla only downloads a maximum of 9998 files per map? Very annoying if you have maps with more than 12000 files. Can this limit be changed somewhere?
Regards,
Hans
I still can find the links to these images in my SQL database tables, and they seem OK. But the images are now corrupted or no longer viewable for some reason. After some googling I read that Filezilla could be responsable for corrupting these images. They weren't downloaded binary or something. But I had selected 'Automatic' for data transfer by Filezilla? I also tried to restore older backups, but this doesn't help. I tried to upload the images with another FTP program without succes. I even tried to fix these images with software like Pixrecovery or PhotoRescue, but alas... Are my images lost now? Any info about how to restore my lost or corrupted images would be very welcome!
Another problem I noticed is that Filezilla only downloads a maximum of 9998 files per map? Very annoying if you have maps with more than 12000 files. Can this limit be changed somewhere?
Regards,
Hans
Re: Problem with corrupted images
If they somehow have been transferred as ASCII type, then they are probably destroyed. ASCII is only for text files because it will perform line ending conversions if necessary as described in Data Type.Are my images lost now?
FileZilla has no limit whatsoever, it has been tested with directories containing millions of files.Another problem I noticed is that Filezilla only downloads a maximum of 9998 files per map? Very annoying if you have maps with more than 12000 files. Can this limit be changed somewhere?
The limit you encounter is on the server. Some hosters intentionally limit (read: cripple) their servers. In your case the limit is 10000 - 9998 files and the two special directories "." and "..".
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
-
- 500 Command not understood
- Posts: 3
- Joined: 2010-05-21 09:02
- First name: Hans
- Last name: Vleugels
Re: Problem with corrupted images
OK, but how come they were transferred in ASCII type when Auto was always selected as option? Could it be because of the filename with no extension?boco wrote:If they somehow have been transferred as ASCII type, then they are probably destroyed. ASCII is only for text files because it will perform line ending conversions if necessary as described in Data Type.
OK, I will check this with my host...boco wrote:FileZilla has no limit whatsoever, it has been tested with directories containing millions of files. The limit you encounter is on the server. Some hosters intentionally limit (read: cripple) their servers. In your case the limit is 10000 - 9998 files and the two special directories "." and "..".
Thx for your help!
Re: Problem with corrupted images
Yes. Filenames without extensions are treated as ASCII files by default. I already said this is a dangerous default setting for that option. Open FileZilla settings, navigate to Transfers - File types, and disable "Treat files without extension as ASCII file".OK, but how come they were transferred in ASCII type when Auto was always selected as option? Could it be because of the filename with no extension?
@botg: That option should be OFF by default. While it may only cause a little inconvenience for some users when it is OFF instead of ON, it irreparably destroys binary files without extensions when it is ON instead of OFF.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
-
- 500 Command not understood
- Posts: 3
- Joined: 2010-05-21 09:02
- First name: Hans
- Last name: Vleugels
Re: Problem with corrupted images
OKay, now I know what happened to my files. Too bad I lost them, but maybe this can be a warning for other users so they don't get the same problem...
-
- 500 Command not understood
- Posts: 2
- Joined: 2010-08-27 02:01
- First name: frank
- Last name: garrett
Re: Problem with corrupted images
I just had this EXACT same problem with Filezilla and Phpbb3. ALL of my pictures are gone also, 6 months of hard work in the trash. I am going to change my settings now. (and YES! you should change the default on new releases.)
BUT why are the binary files PERMENENTLY destroyed? What was done to them was an automated conversion, WHY can't they be converted back? Please explain.
FRANK (fdg123)
BUT why are the binary files PERMENENTLY destroyed? What was done to them was an automated conversion, WHY can't they be converted back? Please explain.
FRANK (fdg123)
Re: Problem with corrupted images
Text files have a well defined structure:BUT why are the binary files PERMENENTLY destroyed?
1. Line of normal text containing printable characters only;
2. Line ending, containing CR+LF (Windows), LF (*NIX, newer Mac), CR (older Mac);
3. Repeat the above until EOF (End Of File)
The conversion simply changes line endings from one version to another.
Example: Uploading a text file from Win86 to *NIX removes all CR characters from the line endings. Downloading the same file back adds the CRs again.
This conversion only works because in text files, the only non-printable characters are the line endings $0D and $0A (and sometimes the EOF character $1A). That makes Word docs, RTF and PDF binary files, they aren't text files!
But image files contain a wild mixture of any available chars, including the line ending characters. Now imagine what happens if all CR characters are removed. File corrupt. And it is destroyed permanently, because converting back would add CRs in places where previously weren't any.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
-
- 500 Command not understood
- Posts: 2
- Joined: 2010-08-27 02:01
- First name: frank
- Last name: garrett
Re: Problem with corrupted images
Ok, now I understand. but thanks for explaining it to me.
FRANK
Experiance is a hard teacher.
FRANK
Experiance is a hard teacher.
-
- 500 Command not understood
- Posts: 1
- Joined: 2013-04-21 03:07
- First name: peter
- Last name: warner
Re: Problem with corrupted images
Saved my bacon... After update, my preferences went back to all ASCII, didn't realize I had to change them again. It's my wife's site I was working on too
Re: Problem with corrupted images
Preferences/settings shouldn't change on their own. Did all settings get reset?
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
- audiopro
- 226 Transfer OK
- Posts: 295
- Joined: 2013-03-23 12:55
- First name: Keith
- Location: Morecambe, England
Re: Problem with corrupted images
I don't understand how this happened to you.fdg123 wrote:I just had this EXACT same problem with Filezilla and Phpbb3. ALL of my pictures are gone also, 6 months of hard work in the trash.
You uploaded the files from your local computer to the server - found they were all corrupted because you transferred them in ascii mode by mistake.
Why didn't you just upload them again but this time as binaries?
As for 6 months work in the trash - backups are essential for all important files.
Morecambe - Where the sun goes at night
Re: Problem with corrupted images
I've created an account here just to say how I fixed my images. This is the first real Google search result so I'd like the next person to end up here to see the solution.
In my case the files corrupted when transferring from a Linux system to Windows.
As I learned from this topic that automagically replaced all \n with \r\n.
What I did was open all image files in the files directory with Notepad++.
Then used the replace function (Ctrl+H) to replace all \r\n in all files with \n. Search mode setting need to be set to Extended.
Save all files and that's it - images look great again.
I also changed the FileZilla setting not to treat files without extensions as ASCII to avoid this issue in the future.
In my case the files corrupted when transferring from a Linux system to Windows.
As I learned from this topic that automagically replaced all \n with \r\n.
What I did was open all image files in the files directory with Notepad++.
Then used the replace function (Ctrl+H) to replace all \r\n in all files with \n. Search mode setting need to be set to Extended.
Save all files and that's it - images look great again.
I also changed the FileZilla setting not to treat files without extensions as ASCII to avoid this issue in the future.
Re: Problem with corrupted images
If that worked you were really lucky. The line ending conversion replaced \n with \r\n - but what about places where there was already \r\n before? I repeat: You were very lucky because your images didn't seem to have such spots.
Of course you can always attempt to do the reverse operation, but you need to be aware that it is far from being reliable. Smaller images = Higher chance.
Of course you can always attempt to do the reverse operation, but you need to be aware that it is far from being reliable. Smaller images = Higher chance.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org