I'm new to the forum, although not to FileZilla. Have been happily using it for quite a few years.
My problem: to update my documentation, I use a "Help Authoring Tool." After making changes, I compile/publish my project and generate about 5000 new files. Now I want to transfer them to the server.
If I use generic "over-write," FZ does the job nicely.
However, only 10/20/50 of my files have changed--i.e., are different from the copies already on the server. I only want to copy those 50.
FZ gives the option to only transfer those that are newer, or only those that have different size. Unfortunately, neither of those options work for me. What I need is some sort of option or process to "transfer only those files that have different contents" process. "Files that are different" are defined as having a different hash or checksum from the versions on the server.
I'm sure I'm not the only person who has faced this situation. Does FZ have any process for handling this?
Thanks for any comments or help.
Transfer only different files
Moderator: Project members
Re: Transfer only different files
That's not possible with the FTP/SFTP protocols, as these do not support direct access to file contents on servers. Only metadata (size or timestamp) is easily available for comparison. Furthermore, with these old protocols, dedicated checksum commands are also absent. As neither the size nor the timestamps will work in your case, overwriting everything for keeping it consistent is the only option, unfortunately.
In other words, for comparing file contents, an FTP client would actually have to re-download every file first, compare locally, then decide if it should be uploaded. This would take much longer and waste more bandwidth than to just upload everything again.
In other words, for comparing file contents, an FTP client would actually have to re-download every file first, compare locally, then decide if it should be uploaded. This would take much longer and waste more bandwidth than to just upload everything again.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
-
- 500 Command not understood
- Posts: 4
- Joined: 2024-04-11 17:58
- First name: Ty
- Last name: Griffin
Re: Transfer only different files
Buco--
Thank you for the in-depth and insightful explanation.
I understand that what I want/need is not possible with pure FTP. But FileZilla is a program that can obviously do much more than issue FTP commands. How about if FZ generated checksums for each local file, put them all into a container file, and sent it to the server. That's the first time. Next time, FZ retrieves the container file from the server and compares it to the newly-generated checksum container file on the local machine. FZ would then know which files had changed, and could then transfer only them. No processing of any sort takes place on the server, only file transfers.
Like I said, I'm pretty sure that other people have encountered the same problem/situation as I have. There should be a solution like the one I've described around somewhere.
And if it isn't around, perhaps the FZ developers could create it. ????
Thank you for the in-depth and insightful explanation.
I understand that what I want/need is not possible with pure FTP. But FileZilla is a program that can obviously do much more than issue FTP commands. How about if FZ generated checksums for each local file, put them all into a container file, and sent it to the server. That's the first time. Next time, FZ retrieves the container file from the server and compares it to the newly-generated checksum container file on the local machine. FZ would then know which files had changed, and could then transfer only them. No processing of any sort takes place on the server, only file transfers.
Like I said, I'm pretty sure that other people have encountered the same problem/situation as I have. There should be a solution like the one I've described around somewhere.
And if it isn't around, perhaps the FZ developers could create it. ????
Re: Transfer only different files
Normally, unless a program jumps through a few extra hoops, changing a file changes its modification time. If the systems' clocks are set correctly you can simply transfer everything where the modification time of the source file is newer than the modification time at the target location.
Re: Transfer only different files
@botg: No, if he builds his project again, ALL the timestamps are different, yet, not all files have actually been changed.
No support requests over PM! You will NOT get any reply!!!
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
FTP connection problems? Please read Network Configuration.
FileZilla connection test: https://filezilla-project.org/conntest.php
FileZilla Pro support: https://customerforum.fileZilla-project.org
-
- 500 Command not understood
- Posts: 1
- Joined: 2024-08-01 20:16
- First name: lily
- Last name: molen
- Location: Pairs
Re: Transfer only different files
Regarding your issue, FileZilla indeed offers options like "transfer newer files" or "transfer files with different sizes," but it doesn't directly support transferring files based on different contents or checksums out of the box.
One workaround you could consider is using a tool outside of FileZilla to generate checksums for your files before transferring. Here’s a possible approach:
Generate Checksums Locally: Use a tool like md5sum or sha256sum to generate checksums for your local files.
Generate Checksums on the Server: Do the same on the server for the existing files.
Compare Checksums: Write a script to compare the checksums of the local files with those on the server. Identify which files have different checksums.
Transfer Only Different Files: Use the list of different files to create a batch transfer in FileZilla or another FTP client that supports scripting.
Alternatively, you might consider using a more advanced file synchronization tool that supports checksum comparisons directly. Tools like rsync (for Unix-based systems) can perform such tasks efficiently.
If FileZilla is a must for your workflow, you might suggest this feature to the developers. They are usually receptive to user feedback and might consider it for future updates.
I hope this helps! Let me know if you need further assistance or have any other questions.
One workaround you could consider is using a tool outside of FileZilla to generate checksums for your files before transferring. Here’s a possible approach:
Generate Checksums Locally: Use a tool like md5sum or sha256sum to generate checksums for your local files.
Generate Checksums on the Server: Do the same on the server for the existing files.
Compare Checksums: Write a script to compare the checksums of the local files with those on the server. Identify which files have different checksums.
Transfer Only Different Files: Use the list of different files to create a batch transfer in FileZilla or another FTP client that supports scripting.
Alternatively, you might consider using a more advanced file synchronization tool that supports checksum comparisons directly. Tools like rsync (for Unix-based systems) can perform such tasks efficiently.
If FileZilla is a must for your workflow, you might suggest this feature to the developers. They are usually receptive to user feedback and might consider it for future updates.
I hope this helps! Let me know if you need further assistance or have any other questions.
<Removed for violation of forum rules.>