Rawtherapee: Closing RT is slow on Windows when cache folder contains a lot of files

[heckflosse]

On my win7 I have the RT cache folder on a SSD.
At closing RT scans the folder and needs about 1 ms per 10 files, which sums up to 2 seconds if the max number of cache entries (set to 20000 at my system) is reached.

The following patch does the following (on Windows)

1) It counts the files in cachefolder without getting the file names and the timestamp.
This is about 10 times faster than getting the file names and timestamps, which are only needed when the number of files is greater than the max. number of cache entries. If the number of files is less or equal than the max. number of cache entries, it returns.

2) If the number of files is greater than the max. number of cache settings, it continues with the same processing as before patch.

That means in case 1) closing RT is more than 10 times faster, while in case 2) it's less than 10% slower

Measured with 10082 files in cache folder:
before patch: 960 ms
after patch: 75 ms

diff --git a/rtgui/cachemanager.cc b/rtgui/cachemanager.cc
index 5f73e9e0f..f33a43c96 100644
--- a/rtgui/cachemanager.cc
+++ b/rtgui/cachemanager.cc
@@ -32,7 +32,8 @@
 #include "options.h"
 #include "procparamchangers.h"
 #include "thumbnail.h"
-
+#define BENCHMARK
+#include "../rtengine/StopWatch.h"
 namespace
 {

@@ -339,6 +340,28 @@ Glib::ustring CacheManager::getCacheFileName (const Glib::ustring& subDir,

 void CacheManager::applyCacheSizeLimitation () const
 {
+    BENCHFUN
+#ifdef WIN32
+    // first count files without fetching file name and timestamp.
+    std::size_t numFiles = 0;
+    try {
+
+        const auto dirName = Glib::build_filename (baseDir, "data");
+        const auto dir = Gio::File::create_for_path (dirName);
+
+        auto enumerator = dir->enumerate_children ("");
+
+        while (numFiles <= options.maxCacheEntries && enumerator->next_file ()) {
+            ++numFiles;
+        }
+
+    } catch (Glib::Exception&) {}
+
+    if (numFiles <= options.maxCacheEntries) {
+        return;
+    }
+#endif
+
     using FNameMTime = std::pair<Glib::ustring, Glib::TimeVal>;
     std::vector<FNameMTime> files;

[heckflosse]

I tested the patch also on my Sabayon machine (of course without the #ifdef WIN32).
Measured using 1000 files in cache
Before patch: 10 ms
After patch: 5 ms

Linux seems to be much faster than windows here...

Any objections to push the patch (with or without the #ifdef WIN32) ?

[Floessie]

If you ask me, push it without the #ifdef WIN32...

[Beep6581]

Thinking of just merging dev into release-5.5 and making an rc2 tonight...