Rawtherapee: Improve time to start rt

[heckflosse]

This issue is about to improve the time to start rt.
First patch will follow soon.

[heckflosse]

Ok, here's a first patch which reduces the init time of rt by ~150 ms at my system by using multiple threads for the init phase.

diff --git a/rtengine/init.cc b/rtengine/init.cc
index 41f599b0..056ad6ae 100644
--- a/rtengine/init.cc
+++ b/rtengine/init.cc
@@ -31,7 +31,7 @@
 #include "profilestore.h"
 #include "../rtgui/threadutils.h"
 #include "rtlensfun.h"
-
+#include "StopWatch.h"
 namespace rtengine
 {

@@ -41,25 +41,40 @@ MyMutex* lcmsMutex = nullptr;

 int init (const Settings* s, Glib::ustring baseDir, Glib::ustring userSettingsDir, bool loadAll)
 {
+    StopWatch Stop1("init.cc init");
+#pragma omp parallel sections
+{
+#pragma omp section
     settings = s;
+#pragma omp section
     ProfileStore::getInstance()->init (loadAll);
+#pragma omp section
     ICCStore::getInstance()->init (s->iccDirectory, Glib::build_filename (baseDir, "iccprofiles"), loadAll);
+#pragma omp section
     DCPStore::getInstance()->init (Glib::build_filename (baseDir, "dcpprofiles"), loadAll);
+#pragma omp section

     CameraConstantsStore::getInstance ()->init (baseDir, userSettingsDir);
+#pragma omp section
     ProcParams::init ();
-    Color::init ();
+#pragma omp section
     PerceptualToneCurve::init ();
+#pragma omp section
     RawImageSource::init ();
+#pragma omp section
     if (s->lensfunDbDirectory.empty() || Glib::path_is_absolute(s->lensfunDbDirectory)) {
         LFDatabase::init(s->lensfunDbDirectory);
     } else {
         LFDatabase::init(Glib::build_filename(baseDir, s->lensfunDbDirectory));
     }
-    delete lcmsMutex;
-    lcmsMutex = new MyMutex;
+#pragma omp section
     dfm.init( s->darkFramesPath );
+#pragma omp section
     ffm.init( s->flatFieldsPath );
+}
+    Color::init ();
+    delete lcmsMutex;
+    lcmsMutex = new MyMutex;
     return 0;
 }

[Floessie]

@heckflosse

by using multiple threads for the init phase

Normally there are dependencies between the various instances during the init phase (not speaking of RT but more generally). I trust your experience that you ruled that out. 😄

[heckflosse]

I checked that. Hope, I didn't miss some thing.

[heckflosse]

Currently the HaldCLUT folder is parsed per editor instance. Means when starting rt in SETM mode it's parsed twice (for the batch editor and for the setm editor).

This could be improved the way it's done in lensprofile.cc for the lensfun comboboxes

[heckflosse]

This small patch reduces the start time of rt by another ~60 ms at my system.

$ git diff rtgui/rtimage.cc
diff --git a/rtgui/rtimage.cc b/rtgui/rtimage.cc
index 16e8e328..b24158b0 100644
--- a/rtgui/rtimage.cc
+++ b/rtgui/rtimage.cc
@@ -28,7 +28,7 @@
 namespace
 {

-std::map<Glib::ustring, Glib::RefPtr<Gdk::Pixbuf>> pixbufCache;
+std::map<std::string, Glib::RefPtr<Gdk::Pixbuf>> pixbufCache;

 }

[Floessie]

@heckflosse I get 32 other matches, where a Glib::ustring is used as a std::map<> index. Now that we know how expensive comparing Glib::ustrings is, we should consider reviewing this...

[Hombre57]

Isn't Glib::ustring used on purpose ?

[Floessie]

@Hombre57 It seems so in some places. In others it's debatable, and there are places, where it's compared to or inserted as std::string. Those might be no-brainers IMHO, but I'm not an expert.

[heckflosse]

@Hombre57 Imho for new RTImage({filename}) we can safely replace Glib::ustring with std::string, as we have full control over the used filenames.

[Hombre57]

@heckflosse

as we have full control over the used filenames.

I agree that the filename is plain ascii chars, but it will be concatenated to Glib::ustring for the full path. No problem with that ?

[heckflosse]

@Hombre57 No problem, because the key for the map does not contain the path

[heckflosse]

Another one:

Currently at start of rt the batchtoolpanel creates the exifpanel and the iptcpanel just to remove it later.
This patch doesn't create the exifpanel and iptcpanel for batch mode.

diff --git a/rtgui/batchtoolpanelcoord.cc b/rtgui/batchtoolpanelcoord.cc
index be092fbf..e7672e4c 100644
--- a/rtgui/batchtoolpanelcoord.cc
+++ b/rtgui/batchtoolpanelcoord.cc
@@ -25,31 +25,14 @@

 using namespace rtengine::procparams;

-BatchToolPanelCoordinator::BatchToolPanelCoordinator (FilePanel* parent) : ToolPanelCoordinator(), somethingChanged(false), parent(parent)
+BatchToolPanelCoordinator::BatchToolPanelCoordinator (FilePanel* parent) : ToolPanelCoordinator(true), somethingChanged(false), parent(parent)
 {

     blockedUpdate = false;
-    // remove exif panel and iptc panel
-    std::vector<ToolPanel*>::iterator epi = std::find (toolPanels.begin(), toolPanels.end(), exifpanel);
-
-    if (epi != toolPanels.end()) {
-        toolPanels.erase (epi);
-    }
-
-    std::vector<ToolPanel*>::iterator ipi = std::find (toolPanels.begin(), toolPanels.end(), iptcpanel);
-
-    if (ipi != toolPanels.end()) {
-        toolPanels.erase (ipi);
-    }
-
     if (toolBar) {
         toolBar->setBatchMode ();
     }

-    toolPanelNotebook->remove_page (*metadataPanel);
-    metadataPanel = nullptr;
-    toiM = nullptr;
-
     for (size_t i = 0; i < toolPanels.size(); i++) {
         toolPanels[i]->setBatchMode (true);
     }
diff --git a/rtgui/toolpanelcoord.cc b/rtgui/toolpanelcoord.cc
index ad0bd09b..eb9131ed 100644
--- a/rtgui/toolpanelcoord.cc
+++ b/rtgui/toolpanelcoord.cc
@@ -28,7 +28,7 @@

 using namespace rtengine::procparams;

-ToolPanelCoordinator::ToolPanelCoordinator () : ipc (nullptr), hasChanged (false), editDataProvider (nullptr)
+ToolPanelCoordinator::ToolPanelCoordinator (bool batch) : ipc (nullptr), hasChanged (false), editDataProvider (nullptr)
 {

     exposurePanel   = Gtk::manage (new ToolVBox ());
@@ -70,8 +70,10 @@ ToolPanelCoordinator::ToolPanelCoordinator () : ipc (nullptr), hasChanged (false
     prsharpening        = Gtk::manage (new PrSharpening());
     crop                = Gtk::manage (new Crop ());
     icm                 = Gtk::manage (new ICMPanel ());
-    exifpanel           = Gtk::manage (new ExifPanel ());
-    iptcpanel           = Gtk::manage (new IPTCPanel ());
+    if(!batch) {
+        exifpanel           = Gtk::manage (new ExifPanel ());
+        iptcpanel           = Gtk::manage (new IPTCPanel ());
+    }
     wavelet             = Gtk::manage (new Wavelet ());
     dirpyrequalizer     = Gtk::manage (new DirPyrEqualizer ());
     hsvequalizer        = Gtk::manage (new HSVEqualizer ());
@@ -194,16 +196,20 @@ ToolPanelCoordinator::ToolPanelCoordinator () : ipc (nullptr), hasChanged (false
     toolPanels.push_back (flatfield);

     toolPanels.push_back (coarse);
-    toolPanels.push_back (exifpanel);
-    toolPanels.push_back (iptcpanel);

-    metadataPanel = Gtk::manage (new Gtk::Notebook ());
-    metadataPanel->set_name ("MetaPanelNotebook");
+    if(!batch) {
+        toolPanels.push_back (exifpanel);
+        toolPanels.push_back (iptcpanel);
+        metadataPanel = Gtk::manage (new Gtk::Notebook ());
+        metadataPanel->set_name ("MetaPanelNotebook");
+        metadataPanel->append_page (*exifpanel, M ("MAIN_TAB_EXIF"));
+        metadataPanel->append_page (*iptcpanel, M ("MAIN_TAB_IPTC"));
+    } else {
+        metadataPanel = nullptr;
+    }
     toolPanelNotebook = new Gtk::Notebook ();
     toolPanelNotebook->set_name ("ToolPanelNotebook");

-    metadataPanel->append_page (*exifpanel, M ("MAIN_TAB_EXIF"));
-    metadataPanel->append_page (*iptcpanel, M ("MAIN_TAB_IPTC"));

     exposurePanelSW    = Gtk::manage (new MyScrolledWindow ());
     detailsPanelSW     = Gtk::manage (new MyScrolledWindow ());
@@ -256,7 +262,11 @@ ToolPanelCoordinator::ToolPanelCoordinator () : ipc (nullptr), hasChanged (false
     toiW = Gtk::manage (new TextOrIcon ("wavelet.png", M ("MAIN_TAB_WAVELET"), M ("MAIN_TAB_WAVELET_TOOLTIP"), type));
     toiT = Gtk::manage (new TextOrIcon ("transform.png", M ("MAIN_TAB_TRANSFORM"), M ("MAIN_TAB_TRANSFORM_TOOLTIP"), type));
     toiR = Gtk::manage (new TextOrIcon ("raw.png", M ("MAIN_TAB_RAW"), M ("MAIN_TAB_RAW_TOOLTIP"), type));
-    toiM = Gtk::manage (new TextOrIcon ("meta.png", M ("MAIN_TAB_METADATA"), M ("MAIN_TAB_METADATA_TOOLTIP"), type));
+    if(!batch) {
+        toiM = Gtk::manage (new TextOrIcon ("meta.png", M ("MAIN_TAB_METADATA"), M ("MAIN_TAB_METADATA_TOOLTIP"), type));
+    } else {
+        toiM = nullptr;
+    }

     toolPanelNotebook->append_page (*exposurePanelSW,  *toiE);
     toolPanelNotebook->append_page (*detailsPanelSW,   *toiD);
@@ -264,7 +274,9 @@ ToolPanelCoordinator::ToolPanelCoordinator () : ipc (nullptr), hasChanged (false
     toolPanelNotebook->append_page (*waveletPanelSW,   *toiW);
     toolPanelNotebook->append_page (*transformPanelSW, *toiT);
     toolPanelNotebook->append_page (*rawPanelSW,       *toiR);
-    toolPanelNotebook->append_page (*metadataPanel,    *toiM);
+    if(!batch) {
+        toolPanelNotebook->append_page (*metadataPanel,    *toiM);
+    }

     toolPanelNotebook->set_current_page (0);

diff --git a/rtgui/toolpanelcoord.h b/rtgui/toolpanelcoord.h
index 5a71ef69..15567968 100644
--- a/rtgui/toolpanelcoord.h
+++ b/rtgui/toolpanelcoord.h
@@ -198,7 +198,7 @@ public:
     CoarsePanel* coarse;
     Gtk::Notebook* toolPanelNotebook;

-    ToolPanelCoordinator ();
+    ToolPanelCoordinator (bool batch = false);
     virtual ~ToolPanelCoordinator ();

     bool getChangedState                ()

[heckflosse]

@Floessie

Now that we know how expensive comparing Glib::ustring s is, we should consider reviewing this...

I tried using std::string instead of Glib::ustring for camconst map. That gives a 3x speedup for the camconst init phase. But the speedup is hidden by the multithreaded init phase now.....

[heckflosse]

Here's a patch provided by @agriggio which reduces the time to build the clut combobox for batch editor from 130 ms to 60 ms and for every subsequent editor from 130 ms to 15 ms. There's a minor issue with this patch: The clut combobox in non batch editor now also has an unchanged entry which fortunately does nothing.

diff --git a/rtgui/filmsimulation.cc b/rtgui/filmsimulation.cc
index 716b686e..6751f16c 100644
--- a/rtgui/filmsimulation.cc
+++ b/rtgui/filmsimulation.cc
@@ -62,8 +62,8 @@ bool notifySlowParseDir (const std::chrono::system_clock::time_point& startedAt)
 FilmSimulation::FilmSimulation()
     :   FoldableToolPanel( this, "filmsimulation", M("TP_FILMSIMULATION_LABEL"), false, true )
 {
-    m_clutComboBox = Gtk::manage( new ClutComboBox() );
-    int foundClutsCount = m_clutComboBox->fillFromDir( options.clutsDir );
+    m_clutComboBox = Gtk::manage( new ClutComboBox(options.clutsDir) );
+    int foundClutsCount = m_clutComboBox->foundClutsCount();

     if ( foundClutsCount == 0 ) {
         pack_start( *Gtk::manage( new Gtk::Label( M("TP_FILMSIMULATION_ZEROCLUTSFOUND") ) ) );
@@ -197,27 +197,36 @@ void FilmSimulation::trimValues( rtengine::procparams::ProcParams* pp )

 //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

+
+std::unique_ptr<ClutComboBox::ClutModel> ClutComboBox::cm;
+
+ClutComboBox::ClutComboBox(const Glib::ustring &path): MyComboBox()
+{
+    if (!cm) {
+        cm.reset(new ClutModel(path));
+    }
+
+    set_model(cm->m_model);
+
+    if (cm->count > 0) {
+        pack_start(cm->m_columns.label, false);
+    }
+}
+
 ClutComboBox::ClutColumns::ClutColumns()
 {
     add( label );
     add( clutFilename );
 }

-int ClutComboBox::fillFromDir (const Glib::ustring& path)
+ClutComboBox::ClutModel::ClutModel(const Glib::ustring &path)
 {
     m_model = Gtk::TreeStore::create (m_columns);
-    set_model (m_model);
-
-    const auto result = parseDir (path);
-
-    if (result > 0) {
-        pack_start (m_columns.label, false);
-    }
-
-    return result;
+    //set_model (m_model);
+    count = parseDir(path);
 }

-int ClutComboBox::parseDir(const Glib::ustring& path)
+int ClutComboBox::ClutModel::parseDir(const Glib::ustring& path)
 {
     if (path.empty() || !Glib::file_test(path, Glib::FILE_TEST_IS_DIR)) {
         return 0;
@@ -327,6 +336,11 @@ int ClutComboBox::parseDir(const Glib::ustring& path)
     return fileCount;
 }

+int ClutComboBox::foundClutsCount() const
+{
+    return cm->count;
+}
+
 Glib::ustring ClutComboBox::getSelectedClut()
 {
     Glib::ustring result;
@@ -334,7 +348,7 @@ Glib::ustring ClutComboBox::getSelectedClut()
     Gtk::TreeModel::Row row = *current;

     if ( row ) {
-        result = row[ m_columns.clutFilename ];
+        result = row[ cm->m_columns.clutFilename ];
     }

     return result;
@@ -343,7 +357,7 @@ Glib::ustring ClutComboBox::getSelectedClut()
 void ClutComboBox::setSelectedClut( Glib::ustring filename )
 {
     if ( !filename.empty() ) {
-        Gtk::TreeIter found = findRowByClutFilename( m_model->children(), filename );
+        Gtk::TreeIter found = findRowByClutFilename( cm->m_model->children(), filename );

         if ( found ) {
             set_active( found );
@@ -358,7 +372,7 @@ Gtk::TreeIter ClutComboBox::findRowByClutFilename( Gtk::TreeModel::Children chil
     for( Gtk::TreeModel::Children::iterator it = childs.begin(); !result && it != childs.end(); ++it ) {
         Gtk::TreeModel::Row row = *it;

-        if ( row[ m_columns.clutFilename ] == filename ) {
+        if ( row[ cm->m_columns.clutFilename ] == filename ) {
             result = it;
         } else {
             result = findRowByClutFilename( it->children(), filename );
@@ -370,7 +384,7 @@ Gtk::TreeIter ClutComboBox::findRowByClutFilename( Gtk::TreeModel::Children chil

 void ClutComboBox::addUnchangedEntry()
 {
-    Gtk::TreeModel::Row row = *(m_model->append());
-    row[m_columns.label] = M("GENERAL_UNCHANGED");
-    row[m_columns.clutFilename] = "NULL";
+    Gtk::TreeModel::Row row = *(cm->m_model->append());
+    row[cm->m_columns.label] = M("GENERAL_UNCHANGED");
+    row[cm->m_columns.clutFilename] = "NULL";
 }
diff --git a/rtgui/filmsimulation.h b/rtgui/filmsimulation.h
index 9979f45b..86a37617 100644
--- a/rtgui/filmsimulation.h
+++ b/rtgui/filmsimulation.h
@@ -11,7 +11,9 @@
 class ClutComboBox : public MyComboBox
 {
 public:
-    int fillFromDir (const Glib::ustring& path);
+    ClutComboBox(const Glib::ustring &path);
+    //int fillFromDir (const Glib::ustring& path);
+    int foundClutsCount() const;
     Glib::ustring getSelectedClut();
     void setSelectedClut( Glib::ustring filename );
     void addUnchangedEntry();
@@ -25,11 +27,19 @@ private:
         ClutColumns();
     };

-    int parseDir (const Glib::ustring& path);
+    class ClutModel {
+    public:
+        Glib::RefPtr<Gtk::TreeStore> m_model;
+        ClutColumns m_columns;
+        int count;
+        ClutModel(const Glib::ustring &path);
+        int parseDir (const Glib::ustring& path);
+    };
+
     Gtk::TreeIter findRowByClutFilename(  Gtk::TreeModel::Children childs, Glib::ustring filename );

-    Glib::RefPtr<Gtk::TreeStore> m_model;
-    ClutColumns m_columns;
+    static std::unique_ptr<ClutModel> cm;
+    int count;
 };

 class FilmSimulation : public ToolParamBlock, public AdjusterListener, public FoldableToolPanel

[heckflosse]

This small patch reduces the time to populate the lensfun treestores from 140 ms to 90 ms

diff --git a/rtgui/lensprofile.cc b/rtgui/lensprofile.cc
index 0c3b227a..f96ef1c1 100644
--- a/rtgui/lensprofile.cc
+++ b/rtgui/lensprofile.cc
@@ -586,9 +586,20 @@ LensProfilePanel::LFDbHelper::LFDbHelper()
     lensfunCameraModel = Gtk::TreeStore::create(lensfunModelCam);
     lensfunLensModel = Gtk::TreeStore::create(lensfunModelLens);

+#ifdef _OPENMP
+#pragma omp parallel sections
+#endif
+{
+#ifdef _OPENMP
+#pragma omp section
+#endif
     fillLensfunCameras();
+#ifdef _OPENMP
+#pragma omp section
+#endif
     fillLensfunLenses();
 }
+}

 void LensProfilePanel::LFDbHelper::fillLensfunCameras()
 {

[agriggio]

@heckflosse Ingo, how about this trick instead? It works for single window modes. For multi windows (i.e. editors in their own window), then it keeps the behaviour above. what do you think?

diff --git a/rtgui/filmsimulation.cc b/rtgui/filmsimulation.cc
--- a/rtgui/filmsimulation.cc
+++ b/rtgui/filmsimulation.cc
@@ -62,8 +62,8 @@
 FilmSimulation::FilmSimulation()
     :   FoldableToolPanel( this, "filmsimulation", M("TP_FILMSIMULATION_LABEL"), false, true )
 {
-    m_clutComboBox = Gtk::manage( new ClutComboBox() );
-    int foundClutsCount = m_clutComboBox->fillFromDir( options.clutsDir );
+    m_clutComboBox = Gtk::manage( new ClutComboBox(options.clutsDir) );
+    int foundClutsCount = m_clutComboBox->foundClutsCount();

     if ( foundClutsCount == 0 ) {
         pack_start( *Gtk::manage( new Gtk::Label( M("TP_FILMSIMULATION_ZEROCLUTSFOUND") ) ) );
@@ -117,7 +117,10 @@
 void FilmSimulation::setBatchMode( bool batchMode )
 {
     ToolPanel::setBatchMode( batchMode );
-    m_clutComboBox->addUnchangedEntry();
+    m_clutComboBox->batchMode = batchMode;
+    if (options.multiDisplayMode) {
+        m_clutComboBox->addUnchangedEntry();
+    }
 }

 void FilmSimulation::read( const rtengine::procparams::ProcParams* pp, const ParamsEdited* pedited )
@@ -197,27 +200,67 @@

 //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

+
+std::unique_ptr<ClutComboBox::ClutModel> ClutComboBox::cm;
+
+ClutComboBox::ClutComboBox(const Glib::ustring &path):
+    MyComboBox(),
+    batchMode(false)
+{
+    if (!cm) {
+        cm.reset(new ClutModel(path));
+    }
+
+    set_model(cm->m_model);
+
+    if (cm->count > 0) {
+        pack_start(cm->m_columns.label, false);
+    }
+
+    if (!options.multiDisplayMode) {
+        signal_map().connect(sigc::mem_fun(*this, &ClutComboBox::update_unchanged_entry));
+    }
+}
+
+
+void ClutComboBox::update_unchanged_entry()
+{
+    auto c = cm->m_model->children();
+    
+    if (batchMode) {
+        if (c.size() > 0) {
+            Gtk::TreeModel::Row row = c[c.size()-1];
+            if (row[cm->m_columns.clutFilename] != "NULL") {
+                row = *(cm->m_model->append());
+                row[cm->m_columns.label] = M("GENERAL_UNCHANGED");
+                row[cm->m_columns.clutFilename] = "NULL";
+            }
+        }
+    } else {
+        if (c.size() > 0) {
+            Gtk::TreeModel::Row row = c[c.size()-1];
+            if (row[cm->m_columns.clutFilename] == "NULL") {
+                std::cout << "  removing " << ((void *)this) << std::endl;
+                cm->m_model->erase(row);
+            }
+        }
+    }
+}
+
 ClutComboBox::ClutColumns::ClutColumns()
 {
     add( label );
     add( clutFilename );
 }

-int ClutComboBox::fillFromDir (const Glib::ustring& path)
+ClutComboBox::ClutModel::ClutModel(const Glib::ustring &path)
 {
     m_model = Gtk::TreeStore::create (m_columns);
-    set_model (m_model);
-
-    const auto result = parseDir (path);
-
-    if (result > 0) {
-        pack_start (m_columns.label, false);
-    }
-
-    return result;
+    //set_model (m_model);
+    count = parseDir(path);
 }

-int ClutComboBox::parseDir(const Glib::ustring& path)
+int ClutComboBox::ClutModel::parseDir(const Glib::ustring& path)
 {
     if (path.empty() || !Glib::file_test(path, Glib::FILE_TEST_IS_DIR)) {
         return 0;
@@ -327,6 +370,11 @@
     return fileCount;
 }

+int ClutComboBox::foundClutsCount() const
+{
+    return cm->count;
+}
+
 Glib::ustring ClutComboBox::getSelectedClut()
 {
     Glib::ustring result;
@@ -334,7 +382,7 @@
     Gtk::TreeModel::Row row = *current;

     if ( row ) {
-        result = row[ m_columns.clutFilename ];
+        result = row[ cm->m_columns.clutFilename ];
     }

     return result;
@@ -343,7 +391,7 @@
 void ClutComboBox::setSelectedClut( Glib::ustring filename )
 {
     if ( !filename.empty() ) {
-        Gtk::TreeIter found = findRowByClutFilename( m_model->children(), filename );
+        Gtk::TreeIter found = findRowByClutFilename( cm->m_model->children(), filename );

         if ( found ) {
             set_active( found );
@@ -358,7 +406,7 @@
     for( Gtk::TreeModel::Children::iterator it = childs.begin(); !result && it != childs.end(); ++it ) {
         Gtk::TreeModel::Row row = *it;

-        if ( row[ m_columns.clutFilename ] == filename ) {
+        if ( row[ cm->m_columns.clutFilename ] == filename ) {
             result = it;
         } else {
             result = findRowByClutFilename( it->children(), filename );
@@ -370,7 +418,7 @@

 void ClutComboBox::addUnchangedEntry()
 {
-    Gtk::TreeModel::Row row = *(m_model->append());
-    row[m_columns.label] = M("GENERAL_UNCHANGED");
-    row[m_columns.clutFilename] = "NULL";
+    Gtk::TreeModel::Row row = *(cm->m_model->append());
+    row[cm->m_columns.label] = M("GENERAL_UNCHANGED");
+    row[cm->m_columns.clutFilename] = "NULL";
 }
diff --git a/rtgui/filmsimulation.h b/rtgui/filmsimulation.h
--- a/rtgui/filmsimulation.h
+++ b/rtgui/filmsimulation.h
@@ -11,12 +11,18 @@
 class ClutComboBox : public MyComboBox
 {
 public:
-    int fillFromDir (const Glib::ustring& path);
+    ClutComboBox(const Glib::ustring &path);
+    //int fillFromDir (const Glib::ustring& path);
+    int foundClutsCount() const;
     Glib::ustring getSelectedClut();
     void setSelectedClut( Glib::ustring filename );
     void addUnchangedEntry();

+    bool batchMode;
+    
 private:
+    void update_unchanged_entry();
+    
     class ClutColumns : public Gtk::TreeModel::ColumnRecord
     {
     public:
@@ -25,11 +31,19 @@
         ClutColumns();
     };

-    int parseDir (const Glib::ustring& path);
+    class ClutModel {
+    public:
+        Glib::RefPtr<Gtk::TreeStore> m_model;
+        ClutColumns m_columns;
+        int count;
+        ClutModel(const Glib::ustring &path);
+        int parseDir (const Glib::ustring& path);
+    };
+
     Gtk::TreeIter findRowByClutFilename(  Gtk::TreeModel::Children childs, Glib::ustring filename );

-    Glib::RefPtr<Gtk::TreeStore> m_model;
-    ClutColumns m_columns;
+    static std::unique_ptr<ClutModel> cm;
+    int count;
 };

 class FilmSimulation : public ToolParamBlock, public AdjusterListener, public FoldableToolPanel

EDIT I forgot half of the patch :-)

[heckflosse]

@agriggio Alberto, I pushed all my patches. Feel free to push your patch as soon it's cleaned ;-)

[agriggio]

It seems that Gtk::FileChooserButton has some serious performance issues, especially on windows (but not only there).

Consider the following test program:

#include <gtkmm.h>
#include <stdio.h>
#include <time.h>

class AGFileChooserButton: public Gtk::Button {
public:
    AGFileChooserButton():
        lbl_("(None)"),
        filename_("")
    {
        box_.pack_start(lbl_, true, true);
        Gtk::Image *img = Gtk::manage(new Gtk::Image());
        img->set_from_icon_name("folder", Gtk::ICON_SIZE_BUTTON);
        box_.pack_start(*Gtk::manage(new Gtk::VSeparator()), false, false, 5);
        box_.pack_start(*img, false, false);
        box_.show_all_children();
        add(box_);
        signal_clicked().connect(sigc::mem_fun(*this, &AGFileChooserButton::show_chooser));
    }

private:
    void show_chooser()
    {
#if 1
        Gtk::FileChooserDialog dlg(static_cast<Gtk::Window &>(*get_toplevel()), "test", Gtk::FILE_CHOOSER_ACTION_OPEN);
        dlg.add_button("Cancel", Gtk::RESPONSE_CANCEL);
        dlg.add_button("Ok", Gtk::RESPONSE_OK);
        int res = dlg.run();
        if (res == Gtk::RESPONSE_OK) {
            filename_ = dlg.get_filename();
            lbl_.set_label(Glib::path_get_basename(filename_));
        }
#else
        GtkWindow *parent_window = ((Gtk::Window*)get_toplevel())->gobj();
        GtkFileChooserNative *native;
        GtkFileChooserAction action = GTK_FILE_CHOOSER_ACTION_OPEN;
        gint res;

        native = gtk_file_chooser_native_new ("Open File",
                                              parent_window,
                                              action,
                                              "_Open",
                                              "_Cancel");

        res = gtk_native_dialog_run (GTK_NATIVE_DIALOG (native));
        if (res == GTK_RESPONSE_ACCEPT)
        {
            char *filename;
            GtkFileChooser *chooser = GTK_FILE_CHOOSER (native);
            filename = gtk_file_chooser_get_filename (chooser);
            filename_ = filename;
            g_free (filename);
            lbl_.set_label(Glib::path_get_basename(filename_));
        }

        g_object_unref (native);
#endif
    }

    Gtk::HBox box_;
    Gtk::Label lbl_;
    std::string filename_;
};


int main(int argc, char **argv)
{
    double start = (double)clock() / CLOCKS_PER_SEC;

    auto app = Gtk::Application::create(argc, argv, "org.gtkmm.examples.base");

    Gtk::Window window;

    Gtk::VBox box;
    int n = 10;
    if (getenv("NUMTEST")) {
        n = atoi(getenv("NUMTEST"));
    }
    if (!n) {
        n = 10;
    }

    bool native = getenv("NATIVE");

    for (int i = 0; i < n; ++i) {
        Gtk::Widget *b = NULL;
        if (native) {
            b = Gtk::manage(new Gtk::FileChooserButton("test button", Gtk::FILE_CHOOSER_ACTION_OPEN));
        } else {
            b = Gtk::manage(new AGFileChooserButton());
        }
        box.pack_start(*b);
    }

    window.add(box);
    window.show_all_children();

    double end = (double)clock() / CLOCKS_PER_SEC;
    double time = end - start;

    printf("time: %.3f seconds\n", time);
    fflush(stdout);

    return app->run(window);    
}

Here are a couple of tests. Linux:

$ NUMTEST=30 ./test
time: 0,048 seconds

$ NATIVE=1 NUMTEST=30 ./test
time: 1,311 seconds

Windows 7 (on a VM):

$ NUMTEST=30 ./test
time: 0.093 seconds

$ NATIVE=1 NUMTEST=30 ./test
time: 2.140 seconds

Maybe we should consider implementing our own file chooser button (doesn't seem to be too difficult).

[Hombre57]

@agriggio said :

Maybe we should consider implementing our own file chooser button (doesn't seem to be too difficult).

It is, at least for me. If I'm not mistaken, you have to handle a whole bunch of functions.

[agriggio]

It is, at least for me. If I'm not mistaken, you have to handle a whole bunch of functions.

@Hombre57 maybe you are right, I didn't study the problem in detail yet. But I'll try :-)

[agriggio]

I've pushed a new branch custom-file-chooser-button that uses our own button. Note that I haven't implemented the full Gtk::FileChooser API, but only the functions that are currently used by RT.
It would be great if you could help testing that everything works as expected :-)
Regarding performance, I don't see much difference here (actually, almost no difference at all), but it is also true that I never experienced slow startup times for RT...

[Beep6581]

Compiled and briefly tested, no issues found!

Opening Preferences in dev takes about a second, opening Preferences in custom-file-chooser-button takes about 30% less. I could give exact measurements if you tell me where to insert the timer.
Sabayon Linux
gtk+-3.22.19
gtkmm-3.22.1

[gaaned92]

W10
Though I never really suffered from slow start, I tested:

• start on empty dir seems faster from about 2 sec to about 1 sec.
• issue found : cannot reach the lensfun DB

[agriggio]

@gaaned92

issue found : cannot reach the lensfun DB

I can't see how this can be related to my changes... are you sure it doesn't occur with the latest dev?

[gaaned92]

Of course I tested dev-5.3-69-g8c328caf and it's ok .
I retested and nothing shows up in GUI

[agriggio]

@gaaned92 can you test this binary? https://filebin.net/yszpw83sr43s76ra/rawtherapee-test.exe
Just put it alongside your "normal" rawtherapee.exe from dev and let me know if lensfun works in there (it does for me here)

[heckflosse]

@Beep6581
Here's a patch to measure the start time (not the whole start time, but it includes the FileChooserButtons)

diff --git a/rtgui/toolpanelcoord.cc b/rtgui/toolpanelcoord.cc
index ed84ae8b..52886700 100644
--- a/rtgui/toolpanelcoord.cc
+++ b/rtgui/toolpanelcoord.cc
@@ -25,12 +25,14 @@
 #include "../rtengine/improcfun.h"
 #include "../rtengine/procevents.h"
 #include "../rtengine/refreshmap.h"
+#define BENCHMARK
+#include "../rtengine/StopWatch.h"

 using namespace rtengine::procparams;

 ToolPanelCoordinator::ToolPanelCoordinator (bool batch) : ipc (nullptr), hasChanged (false), editDataProvider (nullptr)
 {
-
+BENCHFUN
     exposurePanel   = Gtk::manage (new ToolVBox ());
     detailsPanel    = Gtk::manage (new ToolVBox ());
     colorPanel      = Gtk::manage (new ToolVBox ());

[Beep6581]

Sabayon Linux
gtk+-3.22.19
gtkmm-3.22.1

ToolPanelCoordinator took 1139 ms
ToolPanelCoordinator took 1006 ms

ToolPanelCoordinator took 898 ms
ToolPanelCoordinator took 806 ms

[heckflosse]

I measured on Win7 with the patch from last post:
before : ~1400 ms
after: ~900 ms

Lensfun still works fine here

Edit: the above timings have been with HaldClut folder non empty.
With HaldClut folder set to an empty folder it's ~60 ms less in both cases.

[gaaned92]

@agriggio when put in the dev build the rawtherapee-test and rawtherapee-74 find the DB. so it's an other problem.
liblensfun.dll was missing.

[gaaned92]

Windows10

ToolPanelCoordinator took 933 ms
ToolPanelCoordinator took 794 ms

ToolPanelCoordinator took 577 ms
ToolPanelCoordinator took 477 ms

And lensfun works fine

[Beep6581]

Lensfun works fine in custom-file-chooser-button in Sabayon as well.

[Beep6581]

@agriggio the only issue I saw was the minor one of the new widgets lacking the outside padding which the old widgets had - see my screenshots above.

[agriggio]

thanks, see if it's better now

[Beep6581]

Not sure what the best issue is, so I'll put this here.

The "Gtk::FileChooserButton performance on Windows" upstream issue, about the Gtk::FileChooserButton being slow, is now here:
https://gitlab.gnome.org/GNOME/gtkmm/issues/27