Slow listing in local directory with large subdirectories

[Tomzi]

Filezilla 3.3.3 on XP SP3 x86, no AV or other background scanner programs running.

There's a directory with only ~1 MB total size of 16 files, and ~40k files of ~146 GB in 40 subdirectories. Filezilla client takes 10-25 seconds to create a listing of that directory. It appears that instead of just listing the current dir's contents, the program also looks at the immediate subdirs. I guess only 1 level down because the listing in the parent directory (of the one in question) doesn't take long.

Even if it is doing something simple, as e.g. counting files or total file sizes for each subdir, it's taking way too long. In Far Commander the latter takes only a couple of seconds (the first time, I think it's not cached).

Is there any way to stop Filezilla from looking inside the subdirectories?

Sidenote: I did my search for similar threads and found some that looked similar but in fact were about either networked drives or AV/security programs causing slowdown. In my case it's neither.

Thank you for creating Filezilla and your attention.

[boco]

You're talking about a remote listing, right? The AV threads are about local listings.

Filezilla client takes 10-25 seconds to create a listing of that directory.

Enable time-stamped logs and look where the delay in sending the listing is.

It appears that instead of just listing the current dir's contents, the program also looks at the immediate subdirs. I guess only 1 level down because the listing in the parent directory (of the one in question) doesn't take long.

FileZilla doesn't look peek into subdirectories. In fact, you should still see a question mark on all subdirs, meaning not visited yet. Note that visited dirs are cached for the session.

Even if it is doing something simple, as e.g. counting files or total file sizes for each subdir, it's taking way too long.

It doesn't do anything like that.

In Far Commander the latter takes only a couple of seconds (the first time, I think it's not cached).

It's FAR Manager, and you can't compare the two, FAR is using LIST, while FileZilla uses MLSD.

So it must be something other. Post a log, please (time-stamped).

[Tomzi]

Local listing, not remote, as in the directory in question resides on my local drive. Sorry if I wasn't clear on that.

As it's happening on a local drive without any FTP activity, I don't think there will be any useful info in the log. In fact, without FTP activity I'm not even seeing a log file.

Yes it's Far Manager, what an embarrassing mistake. And I've been using it for ages.

As for the rest, I believe you that Filezilla isn't looking into the subdirectories, however, considering that the directory itself only has a couple of small files I'm wondering what the reason could be for the slow listing.

I'll report back if I find anything more informative on this issue.

[boco]

What's the file types of the files in the directory? Could it be they have icons in them? I'm asking because FAR does not access the files for displaying icons, but FileZilla does. And while accessing files for extracting icons to display, an overly aggressive antivirus (real time scanner) can cause a massive slowdown - every file will be checked.

[Tomzi]

Mostly Matlab .mat files and and custom files from a microscope-based image acquisition system (not registered in the OS), very few JPG and TXT files. No icons as far as I know. Definitely no antivirus program running in the background.

Oh, and the only reason I mentioned FAR was because in FAR I can roughly measure the time needed to scan the size of subdirectories, and if Filezilla was doing the same scanning than these times could be compared. If no such scanning takes place then FAR is irrelevant.

Couple of things that occured to me:
Maybe the HDD was put to sleep by the OS - although I find this very unlikely as the computer wasn't left idle, at least not each time I experienced a slow listing.

This slow listing happens mostly when Filezilla starts and happens to default to that directory. I also use a firewall (Comodo IS), is it possible that it somehow interfered with Filezilla starting up and the whole issue has nothing to do with directory listing?

Thanks a lot for your effort to help, but I think I should try to narrow down the possible causes a bit more before I come back for more advice. I know I should've done that before posting but at least it helped me think!

[botg]

I also use a firewall (Comodo IS), is it possible that it somehow interfered with Filezilla starting up and the whole issue has nothing to do with directory listing?

Possibly.