Uploaded image files ASCII, not Binary, images dont show.

Come here to discuss FileZilla and FTP in general

Moderator: Project members

Post Reply
Message
Author
JoeSmith
504 Command not implemented
Posts: 6
Joined: 2011-02-19 01:33
First name: joe
Last name: smith

Uploaded image files ASCII, not Binary, images dont show.

#1 Post by JoeSmith » 2011-02-19 01:57

OS: windows 7,
FTP: Using Filezilla,
Webhost that forum is on godaddy, using linux

-link to what the images look like->http://www.hype-clan.com/hypeforum/view ... b8ecb8363a

The Problem:

Moved my mysql database to new host, and moved phpforum files to new host also.
when uploaded the new php files using filezilla, the filezilla setting where:
-Default transfer settings where on AUTO
-treat files without extension as ASCII file ( this was checked )
-Treat dotfiles as ASCII files. ( this was checked )

so uploaded the php files with these settings

and the pictures where there but all white, and when clicked on thumbnail larger white pic came up with a triangle warning sign.

-- so i went on many forums looking for an answer, i got mixed answers, all around the fact that i didnt upload it in Binary mode or i didnt have the -treat files without extension as ASCII file - unchecked.

so i tried different ways up uploaded just the folder that had the images in it, i tried unchecking the treat files without extension as ASCII files, also unchecked the Treat dotfiles as ASCII files, and put Default transfer setting to binary.

this help a lil, as before all the images were white, and after doing the step above, they end up with blocky colors and some tiny blocks you can see the real image.

then i re-uploaded the entire php folder with all the files, in binary and i just got the same thing as above.

i also tried changing some htcacces code but that didnt seem to help either. ( those files are back to the way the original were)

so im stuck. any help would be great. i tried to provide as much infor about the situation as possible, and all the steps i have taken to try and fix it.

im thinking since the last host for the forum was windows based, and the new host on godaddy is linux, that may be an issue but idk.

User avatar
boco
Contributor
Posts: 25330
Joined: 2006-05-01 03:28
Location: Germany

Re: Uploaded image files ASCII, not Binary, images dont show

#2 Post by boco » 2011-02-19 02:18

The forum you use is phpBB. As far as I know attachments in phpBB are stored as files without extension (in the ./files subdirectory). By default FileZilla treats files without extensions as ASCII files. Binary files (such as images) without extension are DESTROYED that way. Note that the setting works in both directions, so you already destroy the attachments when you download the old forum to your local PC.

''Treat files without extensions as ASCII files'' is the first option I disable on a fresh FileZilla installation. In my opinion, it should be disabled by default.
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
### END SIGNATURE BLOCK ###

JoeSmith
504 Command not implemented
Posts: 6
Joined: 2011-02-19 01:33
First name: joe
Last name: smith

Re: Uploaded image files ASCII, not Binary, images dont show

#3 Post by JoeSmith » 2011-02-19 02:41

so theres no possible way to fix this ?

User avatar
boco
Contributor
Posts: 25330
Joined: 2006-05-01 03:28
Location: Germany

Re: Uploaded image files ASCII, not Binary, images dont show

#4 Post by boco » 2011-02-19 05:42

If you still have access to the original files on the old server, re-transfer them with this option OFF (both down and re-up). Leave it off. The destruction due to ASCII line ending conversion itself can not be reversed/fixed, it is a lossy conversion.

@botg: That option being on by default is dangerous!
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
### END SIGNATURE BLOCK ###

JoeSmith
504 Command not implemented
Posts: 6
Joined: 2011-02-19 01:33
First name: joe
Last name: smith

Re: Uploaded image files ASCII, not Binary, images dont show

#5 Post by JoeSmith » 2011-02-19 07:20

no i dont have the original files or i would have done that.

theres gotta be a way, ive seen convert ASCII to Binary things. but havent tried yet.

User avatar
boco
Contributor
Posts: 25330
Joined: 2006-05-01 03:28
Location: Germany

Re: Uploaded image files ASCII, not Binary, images dont show

#6 Post by boco » 2011-02-19 16:04

The problem with binary files having been mangled by ASCII transfers is that all CRLF/LF/CR characters have been converted. A textfile will only contain one set of line endings (CRLF or LF or CR), so conversion is easy and reversible. But as binary files contain all combination of these characters randomly, how to reverse it if they were modified?

The potential software would have to look for each occurrence of one of these characters/character combination and determine if it was once a CR, a LF, or a CRLF sequence. That's very hard to do, only by bruteforcing. Having the original file size and maybe even CRC could be an advantage.
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
### END SIGNATURE BLOCK ###

JoeSmith
504 Command not implemented
Posts: 6
Joined: 2011-02-19 01:33
First name: joe
Last name: smith

Re: Uploaded image files ASCII, not Binary, images dont show

#7 Post by JoeSmith » 2011-02-19 22:15

:( well thanks for you help im going to keep trying, ive posted the same question on about 8 other forums.

and theres no possible way, then ill give up. but i do find something ill let you know.

JoeSmith
504 Command not implemented
Posts: 6
Joined: 2011-02-19 01:33
First name: joe
Last name: smith

Re: Uploaded image files ASCII, not Binary, images dont show

#8 Post by JoeSmith » 2011-02-23 19:10

i have found the solution, on another forum and will share here, this worked in converting the files to be used once again.

Do you have SSH access to your server?

If not can you download and install perl to windows from here: http://www.activestate.com/activeperl/downloads

I have included a modified version of that script here. Create a new folder, in this folder put the following:

1 - the script below saved as unftp.pl
2 - a directory called images which contains all the images you want to convert from ascii
3 - a directory called images2

If you have SSH access to the server and are doing this on the server, just navigate to the directory with the perl script in, and type perl unftp.pl - then hit enter. If you are on windows just double click the perl script itself - a dos prompt should flash up and vanish again and it should be done. Either way, check the images2 folder which should now be full of binary images and see if it has worked.

Code: Select all

#!/usr/bin/perl -w

use strict;
my $chr;
my $prv;
my $cr = chr(13);
my $lf = chr(10);
my $crlf = chr(13) . chr(10);
my @contents;
my $dir;
my $file;
my $newfiledata;

my $iscr = 0;

$dir=opendir("DIR","images");
@contents=readdir(DIR);
closedir(DIR);

foreach $file (@contents){
        if ($file eq "."){ next; }
        if ($file eq ".."){ next; }
        print "converting file: $file   \n";

        open (INP, "<images/$file") || die $!;
        open (OUTF, ">images2/$file") || die ("Cannot open images2/$file to write to it: " .$!);
        $newfiledata="";
        while (read(INP, $chr, 65536) ) {

                $chr = $cr . $chr if $iscr;

                $iscr = 0;
                if (substr($chr, length($chr)-1, 1) eq $cr) {
                        $chr = substr($chr, 0, length($chr)-1);
                        $iscr = 1;
                }

                $chr =~ s/$crlf/$lf/g;
                $newfiledata .= $chr;
        }
        print OUTF $newfiledata;
        close(OUTF);
}
print "Finished converting files\n";

User avatar
botg
Site Admin
Posts: 33236
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse
Contact:

Re: Uploaded image files ASCII, not Binary, images dont show

#9 Post by botg » 2011-02-23 21:38

Unconditionally replacing CRLF with LF is a bad idea. What if there's supposed to be CRLF?

User avatar
boco
Contributor
Posts: 25330
Joined: 2006-05-01 03:28
Location: Germany

Re: Uploaded image files ASCII, not Binary, images dont show

#10 Post by boco » 2011-02-24 00:25

I hate to quote myself but here goes:
The potential software would have to look for each occurrence of one of these characters/character combination and determine if it was once a CR, a LF, or a CRLF sequence. That's very hard to do, only by bruteforcing.
By ''bruteforcing'' I mean, create an output file for each of the possible combination of character sequences, then visually check for the flawless image.
### BEGIN SIGNATURE BLOCK ###
No support requests per PM! You will NOT get any reply!!!
FTP connection problems? Do yourself a favor and read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
### END SIGNATURE BLOCK ###

JoeSmith
504 Command not implemented
Posts: 6
Joined: 2011-02-19 01:33
First name: joe
Last name: smith

Re: Uploaded image files ASCII, not Binary, images dont show

#11 Post by JoeSmith » 2011-02-25 19:58

well i dont know what yall are saying :/ but it worked and im good with it.

knightmb
500 Command not understood
Posts: 1
Joined: 2011-06-04 18:15
First name: Knight
Last name: MB

Re: Uploaded image files ASCII, not Binary, images dont show

#12 Post by knightmb » 2011-06-04 18:19

I've run into a similar situation. I tried the perl script and while it does output the file in the right place, it appears unchanged from the original. I thought the script would generate several "versions" of the image so one could visually inspect to see which came out right. On the plus side, I do have what size the image is suppose to be. Is there a way to add a field to the script that can take the file size and try combinations until it finds one that matches?

MPDano
500 Command not understood
Posts: 1
Joined: 2015-10-24 14:26
First name: Leo
Last name: Dano

Re: Uploaded image files ASCII, not Binary, images dont show

#13 Post by MPDano » 2015-10-24 14:35

JoeSmith wrote:i have found the solution, on another forum and will share here, this worked in converting the files to be used once again.

Do you have SSH access to your server?

If not can you download and install perl to windows from here: http://www.activestate.com/activeperl/downloads

I have included a modified version of that script here. Create a new folder, in this folder put the following:

1 - the script below saved as unftp.pl
2 - a directory called images which contains all the images you want to convert from ascii
3 - a directory called images2

If you have SSH access to the server and are doing this on the server, just navigate to the directory with the perl script in, and type perl unftp.pl - then hit enter. If you are on windows just double click the perl script itself - a dos prompt should flash up and vanish again and it should be done. Either way, check the images2 folder which should now be full of binary images and see if it has worked.

Code: Select all

#!/usr/bin/perl -w

use strict;
my $chr;
my $prv;
my $cr = chr(13);
my $lf = chr(10);
my $crlf = chr(13) . chr(10);
my @contents;
my $dir;
my $file;
my $newfiledata;

my $iscr = 0;

$dir=opendir("DIR","images");
@contents=readdir(DIR);
closedir(DIR);

foreach $file (@contents){
        if ($file eq "."){ next; }
        if ($file eq ".."){ next; }
        print "converting file: $file   \n";

        open (INP, "<images/$file") || die $!;
        open (OUTF, ">images2/$file") || die ("Cannot open images2/$file to write to it: " .$!);
        $newfiledata="";
        while (read(INP, $chr, 65536) ) {

                $chr = $cr . $chr if $iscr;

                $iscr = 0;
                if (substr($chr, length($chr)-1, 1) eq $cr) {
                        $chr = substr($chr, 0, length($chr)-1);
                        $iscr = 1;
                }

                $chr =~ s/$crlf/$lf/g;
                $newfiledata .= $chr;
        }
        print OUTF $newfiledata;
        close(OUTF);
}
print "Finished converting files\n";
I tried this and it didn't work. It wrote perfectly to the images2 directory, but what I uploaded them back to the files directory, they are still broken. Was there any other things to try? 3,000 pictures gonnilla using filezilla.

Post Reply