FileZilla Forums

Sorry if this is an old question but I haven't been able to find a clear answer... can FZ Server generate and return a checksum (CRC/MD5/SHA1,etc) for a sitting file when requested by a remote CLI client? I have a BASH script running on a server that FTP's a couple of nightly database dumps to a Fiilezilla server (v. 0.9.60), and I'd like to be able to verify after the uploads that the uploaded files exist on the FZ server and the files' checksums on the FZ server match those of the original files.

Any suggestions on how I might achieve that?

Only way is to download the files again to a different location and compare source and target.

botg wrote: ↑

2022-05-03 08:02

Only way is to download the files again to a different location and compare source and target.

Thanks, I was considering that for a plan "B."

@botg: What happened to your draft proposal for the "HASH" command?

Expired. Not enough interest in it.

With FTP over TLS the feature is redundant anyhow for the workflow requested in this topic.

There is one easily overlooked aspect of modern computers that make it extremely difficult to actually verify the data on the disk: Caching. You never know whether data you read is fresh from the storage device, or data from a cache. Attempting to control caching behavior not only hurts performance, it is also near impossible, especially if you consider use-cases such as network attached storage. Depending on the storage architecture, to reliably ensure all data is read fresh from storage you would need to power-cycle the server machine and the attached storage systems.

My recommendation for reliable storage:
* Use a checksummed filesystem or block device
* Redundant storage on either filesystem or block level, as long as the redundancy is on the same level as the checksumming
* Uninterruptible power supply
* ECC memory

You forgot:
* Lots of money.

Filesystem with redundancy and checksumming: Free. Multiple ones included in any recent Linux kernel.
USV: El-cheapo USV with puny battery sufficient. Main purpose is to safely shut down the system in case of power loss. 50 bucks.
RAM with ECC: just 30% more than non-ECC RAM of the same capacity, pair it with a recent AMD CPU. You need very little RAM for a file server.

botg wrote: ↑

2022-05-04 08:47

Expired. Not enough interest in it.

With FTP over TLS the feature is redundant anyhow for the workflow requested in this topic.

Apologies for resurrecting this thread but the issue is still ongoing... @botg, I was asking about the HASH function to verify uploaded files and here you say that FTP over TLS makes a HASH function redundant. Does TLS provide integrity checking on transferred files and what happens, then, if a file becomes corrupt during transfer or the transfer is aborted leaving an incomplete file on the destination? Google isn't being very cooperative when I search this question. I would be performing the uploads with whatever PHP functions are necessary for FTP over TLS.

Yes, TLS protects data integrity. Depending on the server used, you may end up with incomplete, but otherwise 100% correct data on the server. As the transfer-initiating part in FTP is the client, it is the client's responsibility to retry interrupted transfers.

I advise against using PHP for FTP. I tried myself a few years back when creating https://ftptest.net/ and found out the hard way that it's impossible to write secure FTP software in PHP. Use a proper programming language such as C or C++ instead.