Imgui: Generic state storage

[rokups]

Could we get a generic state storage? I am aware of ImGui::GetStateStorage() but it is very limited. Main issue with it is that it stores only void*/float/int, but state for complex widgets is far more complex. We end up having to store one field at a time, which is inconvenient, creates code smell and does N-1-too many lookups on every frame.

We could allocate a struct on heap and store it's pointer. This is problematic because we also have to free this state once widget is no longer used. I do not see a way to do it transparently without user ever caring.

We could definitely use a way to store some data (sizeof(data) > sizeof(void*)) whose lifetime is tied to current item id or at least to lifetime of current window.

I have a custom workaround that you will not like because c++ magic, but it might serve as a reference idea to bounce off.

/// Get custom user iu state at given position of id stack. If state does not exist then state object will be created.
/// Using different type at the same id stack position will return new object of that type. Arguments passed to this
/// function will be passed to constructor of type T.
template<typename T, typename... Args>
T* GetUIState(Args... args)
{
    ImGui::PushID(typeid(T).name());
    T* state = (T*)GetUIStateP();
    if (state == nullptr)
    {
        state = new T(args...);
        SetUIStateP(state, [](void* s) { delete (T*)s; });
    }
    ImGui::PopID();
    return state;
}
/// Expire custom ui state at given position if id stack, created with GetUIState<T>. It will be freed immediately.
template<typename T>
void ExpireUIState()
{
    ImGui::PushID(typeid(T).name());
    ExpireUIStateP();
    ImGui::PopID();
}


static ea::unordered_map<ImGuiID, UIStateWrapper> uiState_;
static int uiStateLastGcFrame_ = 0;

void SetUIStateP(void* state, void(*deleter)(void*))
{
    auto id = ui::GetCurrentWindow()->IDStack.back();
    uiState_[id].Set(state, deleter);
}

void* GetUIStateP()
{
    void* result = nullptr;
    auto id = ui::GetCurrentWindow()->IDStack.back();
    auto it = uiState_.find(id);
    if (it != uiState_.end())
        result = it->second.Get();

    int currentFrame = ui::GetCurrentContext()->FrameCount;
    if (uiStateLastGcFrame_ != currentFrame)
    {
        uiStateLastGcFrame_ = currentFrame;
        for (auto jt = uiState_.begin(); jt != uiState_.end();)
        {
            if (jt->second.IsExpired())
            {
                jt->second.Unset();
                jt = uiState_.erase(jt);
            }
            else
                ++jt;
        }
    }

    return result;
}

void ExpireUIStateP()
{
    auto it = uiState_.find(ui::GetCurrentWindow()->IDStack.back());
    if (it != uiState_.end())
    {
        it->second.Unset();
        uiState_.erase(it);
    }
}

Basically GetUIState<UserType>() will allocate a new struct of UserType and return it, upon any subsequent calls it same call will return previously allocated struct. However if this struct is not requested for some time it will be deleted. Perfect for temporary widget state. Could we get something like that in ImGui? Or am i doing it totally wrong and there is a better way?

[ocornut]

Hello,

You literally just wrote it. My question is: why does it has to be in ImGui and not in your own codebase?
Where do we draw the line and dig into Dear ImGui being a larger framework?

My intuiton is that my probably don't need any of this in core imgui,
I wouldn't mind a separate helper repo (or use imgui_club) if for some reason it fit particularly well in the imgui mindset, and preferably it wasn't maintained by me.

Please also note the existence of e.g. ImPool in imgui_internal.h which may also be useful somehow.

[rokups]

Reason i am suggesting this for ImGui is because it is a rather basic building block for custom UI widgets and everyone wishing to store custom widget state would find it useful. I am in the process of writing (yet another) node graph and such thing would be very useful. I could include this solution as part of node graph. Same thing is already part of my other project in which i intend to use node graph. Then someone else would make their custom widget and possibly add some kind of their own solution and so on.. It starts to get stupid.

A separate project could almost work too. Problem is that we cant make it 100%-transparent to the user either. I imagine required state could be stored in ImGui::GetStateStorage() or some per-context variant of that (if such thing even exists). Thing is that ImGui::Shutdown() will likely leak memory. Ability to run some code on window deletion or context shutdown would be helpful in this case.

[ocornut]

Reason i am suggesting this for ImGui is because it is a rather basic building block for custom UI widgets and everyone wishing to store custom widget state would find it useful.

But there are infinite ways to store data, people can use some sort of key-value STL container.

Managing those 50+ lines in 2-3 of your private projects is magnitude more simple than me designing and maintaining a public facade that people are relying into. Also the general solution is likely going to be more complex.

If anything, I should try to reduce the API surface to a minimum and being distracting myself with non-priority features, so you'll have to convince me with a great design and great value.

I'm totally open to anything that could help with solving implementation problem (e.g. simple helper for storing void* in imgui context for constant time user access, or to avoid leaking on Shutdown etc.), but may not have time to design or implement those either.

[rokups]

I understand. What would be very convenient is extending ImGuiStorage::SetVoidPtr() to also take an optional deleter function. That way we could store anything we want there and be sure things would get deallocated properly. That would expand size of pair by a third though, and deleter pointer would be unused most of the time so i am not sure that is acceptable. Although if we added ImVec2 storage support to ImGuiStorage then void*+deleter* is much easier justified. Another thing that bothers me about this solution is that storage is per-window. When window is closed Begin() returns false and thus our custom widgets will not render. That will not give a chance to expire our custom state if that would be desired. I may be overthinking it.

[nem0]

sizeof(ImVec2) is 8, 2 * sizeof(void*) is 16

[ocornut]

Why not just using a ImGuiStorage stored in your library and handle destruction yourself at whatever scope/time you want? That's probably even less code that using the thing you suggested, while not putting extra burden on core ImGui.

[rokups]

The whole point of this request is to make it convenient for user to store complex state easily while allowing user to not care about those details. Of course we can do it all manually.. However this kind of state is very much implementation detail that i would rather not have users worry about.

I would very much prefer user to have to do this:

ImGui::CreateContext();
MyFancyLib::MyFancyWidget();
ImGui::DestroyContext();

Than this:

ImGui::CreateContext();
MyFancyLib::Init();
MyFancyLib::MyFancyWidget();
MyFancyLib::Deinit();
ImGui::DestroyContext();

But if you still think this is pointless - feel free to close the issue.

[ocornut]

It is certainly not pointless at all, but the tradeoff seem a little misled here, especially if (and I understand that's a simplification) the gain is mostly to avoid calling an Init/Destroy function on your library.

I am literally drowning in requests, bug to fixes and urgent features so I need to aggressively fend off requests for non-essential features because all the cognitive cost will fall on me forever.

I think it's preferable to keep that sort of thing inside multiple private projects as long as you can, if only to learn more about different use cases. When you've got multiple projects using something you'll probably have a better vision of how to add this to core imgui with minimal impact. The solution probably won't be to just add function pointers for destruction, there are countless ways to handle lifetime and provide context to said functions.

For example, maybe adding the equivalent of an "atexit", aka registering handlers to be called during context destruction, may be just simpler?

[rokups]

For example, maybe adding the equivalent of an "atexit", aka registering handlers to be called during context destruction, may be just simpler?

That sounds good enough. Can windows be permanently removed? If yes - we would also need similar callback to handle closing of the window since storage is per-window.

I can also submit a PR. Just wanted to figure out acceptable way to handle this and avoid PR being rejected :)

[ocornut]

Windows currently cannot be permanently removed but I would like to add support for that in the future. Maybe we'll later add some kind of callback to get notified of removed windows. I think it'll be ok if removing a window is a costly gc-ish like operation, it is not meant to be happening frequently. More like a "clear out unneeded resources".

Just wanted to figure out acceptable way to handle this

Figuring that out is still taking time (not trying to be rude, but I need to point out that every request for feedback and review are somehow distracting me from advancing important features).

I still believe you should strive to use storage in more projects before we add something to imgui public api that comes with any long term guarantee, because right now your case short-term seem to be to only be to avoid user calling a single Shutdown function (which probably wouldn't even be problematic if it was forgotten by the user).

[thedmd]

For example, maybe adding the equivalent of an "atexit", aka registering handlers to be called during context destruction, may be just simpler?

@ocornut
This is literally what I was looking for. Do you want a PR for that or this is something in the "bright future" category?

[thedmd]

I'm trying to use ImGuiSettingsHandler to store node editor data. I have some troubles with it:
1) Handler implicitly require from me to duplicate window settings approach, prepare some cache to deffer data until my widget is ready.
2) Handler lifetime is tied to ImGui context, but my widget has no way of telling if context is being freed. So my handler has to be static, global and thread save in case multiple ImGui instances are created on different threads. This is a heavy burden.
3) After I install new handler it will be feed with data only if I manage to do that before first frame.
4) After my widget is gone and handler uninstalled, next save will wipe out my data.
5) There is no explicit API to install/uninstall handler.

Am I missing something obvious about intended usage of ImGuiSettingsHandler?

In the mean time I would like to relate this comment to whole discussion. I think some problems I have can be addressed by adding new callback function info ImGuiSettingsHandler:

    void        (*CloseFn)(ImGuiContext* ctx, ImGuiSettingsHandler* handler);      // Close: Context is being finalized, there is your chance to free your data

With this function I can:

• free my widget resources related to this context.
• hold pointer to some non-static data structure in ImGuiSettingsHandler::UserData
  I think this will address 1) and 2).

Ad. 3) Shoud I manually call LoadIniSettingsFromDisk()? In case when ini filename isn't set I will have to use LoadIniSettingsFromMemory(). When it is set ImGui is in control. What if I call load when save is pending (SettingsDirtyTimer is sill ticking)?

Can LoadIniSettingsFromDisk() be called after installing new handler only for this one handler? That would require adding explicit internal function like ImGui::InstallSettingsHandler() which can do that. As a result new handler will be feed with data and pending save will not be override.

Ad. 4) I don't think something can be done about that. ImGui does not cache content of the ini file nor provide some kind of persistent key/value storage. What I can do is to make sure handler is tied to context lifetime (or is static).
ImGui actually always has a copy of ini in SettingsIniData. Having that data from no longer known handlers can be left intact.

Ad. 5) ImGui::InstallSettingsHandler() and ImGui::UninstallSettingsHandler() can be added to avoid user code from touching context fields directly. First one can also be utilized as mentioned in Ad. 3).

I can test these ideas on my code and make a PR. Before I do that please let me know if I managed to miss something obvious and what are you thoughts.

[ocornut]

@thedmd Sorry I haven't been answering this, I just stumbled on this again while reviewing some of the still-open issues. It is still something you wish to push ahead?

• The reason for (1) as you may I have guessed is to facilitate storage of persistent config data that are not used during a specific session.

• CloseFn may better be named e.g. OnShutdownFn ? to avoid confusion with the word Open in ReadOpenFn ?

• Not sure how/if (3) is a problem? It is because your subsystem is dynamically loaded later in the app lifetime and you don't have a "startup init" ? It seems wildly simpler to just register that handler before the first frame. We could maybe even consider locking the handler list after the first frame?

• (4) is symmetrical with (3), handlers were designed to have the same lifetime as the imgui context. If the reason for your (3) is that you don't have a "on startup init" then you could leave that running. However note that SettingsIniData will be cleared on the next save, so your handler would eventually need have same lifetime as the imgui context.

• (5) I don't have the install function but the existence of UninstallSettingsHandler is also overriding the idea that handlers need to be the same lifetime.

I am open to PR for that new close/shutdown function, and explanation about why you need to decorrelate the lifetime of settings handle from the context, and how we could even do that (right now I don't have answers for 3-4-5).

As a tangential topic, I thought we could add a system to store user pointers in the imgui context.
Say: imgui context allows to register void* and return an handle (e.g. an index) then it could facilitate subsystems linking their data to the context.

[staticocean]

General words and thank you
@ocornut Hello! Having a blast using ImGui. Currently I am working on a scientific application and an industrial control panel both using ImGui as a main UI library.

The problem
I have some custom widgets (for example 3D plot view). I need to store 3x3 matrices between frames and I do not want to externally allocate gui structure to store it. So the question is: what is the preferred and cleanest current way to store custom structures between frames? Has anything changed? Are there plans to implement this feature?

My current solution
I have a library which does the job and I create UI on top of it.

typedef struct {
    int obj_property;
} obj;

typedef struct {
    int gui_property;
} obj_gui;

void obj_view(obj_gui *gui, obj *object) {
    // do UI stuff
}

What I would like

typedef struct {
    int obj_property;
} obj;

typedef struct {
    int gui_property;
} obj_gui;

void obj_view(obj *object) {
    obj_gui default_gui;
    obj_gui *gui = ImGui::GetState(object, default_gui);

    // do UI stuff
}

[rokups]

There is no one-size-fits-all solution. For example if there is a finite set of GUI properties, then it would be best to just store those properties manually somewhere (class, global scope). I created this issue because i needed to attach some state to various elements in tree-like structures of unknown size. In that case best you can do is allocate some memory and attach it to current ID at the back of ID stack. Of course then you have a hashmap lookup every time element is rendered. Extremely large UIs would need extra optimizations. Seems like my example code in first issue post is close to what you need. You could adapt it to your application easily.

Reason this functionality is not in the core library is that it is not clear whether we can come up with a solution that fits needs of everyone. My snippet, for example, takes some rather specific decisions regarding memory management. Some users may not be OK with using new, they may need to use a custom allocator. Also garbage collection in my code runs when item is accessed. That may or may not be acceptable. Someone may need to keep some states indefinitely for example, or modify expiration time, or expire them at the end of frame instead of during access. There are lots of open questions, therefore it would be best to implement this functionality as part of your application for now.