Feature Request: Temporary File Names for items in Transit

Come here to discuss FileZilla and FTP in general

Moderator: Project members

Post Reply
Message
Author
MrBurritoman
504 Command not implemented
Posts: 8
Joined: 2012-01-24 00:43
First name: MrBurrito
Last name: Man

Feature Request: Temporary File Names for items in Transit

#1 Post by MrBurritoman » 2016-09-14 22:54

Hello,

I have seen this topic a few different times and each time don't see anyone fully flesh out a way to accomplish what it seems several people trying to do.
Here:
viewtopic.php?t=21880
Here 2:
viewtopic.php?t=14694

Not wanting to raise one of these posts from the dead I hope a new one may be helpful.

My regular use for FileZilla is to transfer backup images for our customers to our data center when the normal replication software becomes backed up or has issues. When transferring these backup images I have an issue. The backup replication destination in our data center is a folder that is monitored by software that looks for new files and tries to verify it.

The software the vendor provides to replicate these images uses FTP as its method and while the files are in transit they have the added extension .tmp until the transfer is complete. When the transfer is completed with the vendor's software the file is renamed and the .tmp extension is removed. When this occurs the monitoring software sees the new backup image and knows nothing else is touching it.

When I utilize FileZilla to try and transfer the file the monitoring software doesn't understand that something else may be locking the file and it freaks out because it doesn't match the md5 sum that it is expecting to see.

I know this feature will be useful to many other people outside of my scenario above, web developers or others who have automated processes looking for files to show up.

Would the following methods be possible with FileZilla as a built-in mini script of sorts?
Standard Transfer:
1. Create filename.tmp in the destination and transfer the data to it.
2. Transfer complete, rename operation to remove .tmp from the end of the file.

Overwrite transfer:
1. Create a filename.tmp in the testination and transfer the data to it.
2. Transfer complete, compare and use set logic to overwrite, rename, skip, etc.
2a. If overwrite is selected, delete the original file, Rename the filename.tmp to remove the .tmp
2b. If rename is selected rename the filename.tmp to the new filename.new
etc.

Is there anything I am missing here?

Am I the only one who wants this feature?

I really hate using other FTP applications because most of them are terrible or laden with other software that I don't want to deploy to customer's servers.

User avatar
botg
Site Admin
Posts: 35509
Joined: 2004-02-23 20:49
First name: Tim
Last name: Kosse

Re: Feature Request: Temporary File Names for items in Transit

#2 Post by botg » 2016-09-15 08:01

This feature is a lot more complex than it might seem at first glance.

Any implementation of this must deal with the following issues:

- How to handle naming collisions where the temporary file name already exist?

- How to prevent littering the server with temporary files in case of errors, e.g. when the server does not support rename? If it does not support rename it might not even allow delete either.

- How to deal with server-side restrictions on valid filenames?

MrBurritoman
504 Command not implemented
Posts: 8
Joined: 2012-01-24 00:43
First name: MrBurrito
Last name: Man

Re: Feature Request: Temporary File Names for items in Transit

#3 Post by MrBurritoman » 2016-09-15 08:15

Well, let me take a crack at some of these issues.
botg wrote:...
- How to handle naming collisions where the temporary file name already exist?
Offer two options in the settings:
1. Over write temp file conflicts.
2. Create a new temp suffix for the conflicting file (i.e. .tmp1, or something similar).
botg wrote:...
- How to prevent littering the server with temporary files in case of errors, e.g. when the server does not support rename? If it does not support rename it might not even allow delete either.
Before copying a batch of files from the queue send a test file and see if the rename works on the remote server. If it does not work prompt the user and recommend turning off the temporary file name settings for:
1. That particular server
2. This and all future servers.
botg wrote:...
- How to deal with server-side restrictions on valid filenames?
Somewhat the same as above, this can be part of a small file test to determine if the .tmp filename is allowed, if not then prompt, same as above.

dvdmrrck
500 Syntax error
Posts: 12
Joined: 2009-02-10 12:31
First name: David
Last name: Merrick

Re: Feature Request: Temporary File Names for items in Transit

#4 Post by dvdmrrck » 2020-06-10 08:05

This is really the most needed feature of all in Filezilla. When you're uploading a file, you don't want to put the existing files out of action, or lose them on an aborted upload, and the easiest way is for filezilla to upload the file as a temp and then rename it to the destination name. This is particularly true of larger files which take a little bit to upload as tiny files upload near-instantly.
This could be tackled in settings where you can provide a prefix/suffix pair that get added to the filename for upload and then the file renamed to its destination with associated replacement method option (default: detect - filezilla could easily detect first time if rename replace fails and so do a delete rename and remember what decision it took). So if a prefix/suffix is/are not entered, it just uploads directly, so people will only enter them if it works out for them, which would for most. If there is a standard max filename length on systems, then a filename over that length should use the direct upload method.
Yes, aborts may leave scattered debris but users can accept and deal with that if they want this feature, I'd rather have debris than out of action files, and actually since filezilla knows what it has been uploading and whether it got to rename it, it can record and make use of that fact. Equally, on making a directory list filezilla could detect any files matching the prefix/suffix pattern and alert the user and ask if they should be removed (an option can be provided that this action be done automatically).
Can I recommend that this be given a high priority with some basic implementation as users can choose not to fill in the prefix/suffix pair and as said I think most people wanting such a via-temp upload system would rather have something even if it means a bit of debris than not have it.
Cheers very much,
David

Post Reply