Replace existing

Come here to discuss FileZilla and FTP in general

Moderator: Project members

Post Reply
Message
Author
MalcolmStorey
500 Command not understood
Posts: 2
Joined: 2024-04-18 12:46
First name: Malcolm
Last name: Storey

Replace existing

#1 Post by MalcolmStorey » 2024-04-18 12:58

My internet connection just dropped for ten minutes in the middle of an upload, so I restarted FileZilla. (Maybe I didn't need to but initially thought the upload site had gone down).

Anyway, on rerunning the upload FZ spotted some of the files already exist and asked whether to skip, overwrite or compare date and length. I opted for compare date and length.

But I'm left wondering if this is completely safe. Without knowing the inner workings of FZ or FTP in general, I dunno if files are written in the normal way, growing as necessary, or if it tries to be clever by allocating the space then filling it. In the latter case comparing date and length could still leave you with half-empty files (and I guess also in the former as files grow a block at a time).

Or maybe files are created as .tmp then renamed when complete, in which case it's fine, but I think FTP predates file extensions.

(FTP would be so much more reliable if the original spec had allowed a checksum check.)

User avatar
boco
Contributor
Posts: 26949
Joined: 2006-05-01 03:28
Location: Germany

Re: Replace existing

#2 Post by boco » 2024-04-18 13:35

FTP writes files as stream, continuously. The written data is Byte-precise, please don't confuse allocation with actual data written. A file can be of size 2019 Bytes (actual data) on your PC but its allocated size is actually 4096 Bytes. The slack space is never handled as actual data, it's just overhead.

Note that both date and size comparison are not 100% reliable as the client needs to rely on the data sent by the server.
- Size can only be used for binary files, as text files will have a different size in some cases due to line ending conversion. See Data Type. Plus, servers might not even report the correct size.
- Timestamp comparison only works if the server supports MFMT, the "Preserve timestamps" option in FileZilla is enabled, and the server reports timestamps correctly in listings.

Another "Overwrite" method is Resume. It simply continues writing the data to the file from an offset the server reports. With a proper server, this is perfectly safe. Temporary files that are renamed at the end are not part of the FTP specifications, and such behavior would make resuming, date and time comparison impossible, as the client would not even find the file after reconnection.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org

MalcolmStorey
500 Command not understood
Posts: 2
Joined: 2024-04-18 12:46
First name: Malcolm
Last name: Storey

Re: Replace existing

#3 Post by MalcolmStorey » 2024-04-18 13:49

Thanks.
It's a linux server. W11 at my end. I'm uploading 35K htm files, so text, yes. (35K is no of files, not file-size - they're mostly about half that - total 440mb.)

So if I say "compare date and length" with htm files, will it just compare date or always overwrite? (I suspect the latter cos it seems to have started again without whizzing thru the first few thousand.)

User avatar
boco
Contributor
Posts: 26949
Joined: 2006-05-01 03:28
Location: Germany

Re: Replace existing

#4 Post by boco » 2024-04-18 16:33

ASCII files (aka text aka TYPE A files) will be always overwritten, as it cannot reliably detect if they are same or different. There is no other way.

FileZilla does neither allow comparison nor resuming of ASCII files for that reason. Unfortunately, your case is one of the worst case scenarios for FTP (text files, and small files causing significant transfer overhead). Large binary files is where it really shines.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org

Post Reply