Sqlite_orm: Document compatibility with Sqlite3 compile time options

[MSonn]

Hi fnc12,

i recently had to deal with a application crash and a custom sqlite3 build. The reason was the SQLITE_OMIT_AUTOINIT define that resulted in Sqlite3 not automatically calling sqlite3_initialize and as it turned out sqlite_orm does not call this function either. It probably would be useful if this was documented as this is one of the recommended sqlite3 compile options:

https://www.sqlite.org/compile.html

[fnc12]

Hi @MSonn . I don't understand how compile time options docs in this repo can solve your runtime error. Compile time options affect your libsqlite3 not sqlite_orm. And sqlite_orm is planned to work in both cases: when you have SQLITE_OMIT_AUTOINIT defined and when you have it not defined. I understand that this repo is not well documented but docs will not help you to avoid a crash.

[MSonn]

The basic idea is that it is documented how variant X of sqlite3 interacts with sqlite_orm. If possible also how a variant of sqlite3 will interact with future version of sqlite_orm e.g. you are planning to support SQLITE_OMIT_AUTOINIT. It would not avoid an error in most cases (compile time or runtime) due to the design of sqlite3, however a set of preconditions (here compile time options of Sqlite3) would help a user to verify that "his" libsqlite3 is compatible with sqlite_orm without knowing the internals of sqlite_orm and/or testing it.

Another example would be SQLITE_OMIT_DEPRECATED: A user could for example ask if sqlite_orm depends on the deprecated API and what part of sqlite_orm would not work if this option was set. Similar all remaining SQLITE_OMIT_X Options that remove functionality of the API e.g. SQLITE_OMIT_PROGRESS_CALLBACK, SQLITE_OMIT_DECLTYPE. Other options change the API e.g. SQLITE_DQS.

[fnc12]

sqlite_orm is a frontend for SQLite3. Assume you use raw libsqlite3 without sqlite_orm. Then you would receive the same error. If so it means that sqlite_orm has nothing to do with that cause the problem is on a client side (yours). On the other hand if behavior differs with sqlite_orm and without then it means that error is located inside sqlite_orm.

[fnc12]

I can add functions that will call sqlite3_initialize, sqlite3_shutdown, sqlite3_os_init and sqlite3_os_end like this:

sqlite_orm::initialize();
sqlite_orm::shutdown();
sqlite_orm::os_init();
sqlite_orm::os_end();

And then you will call them by yourself depending on whether your sqlite is compiled with or without SQLITE_OMIT_AUTOINIT. BTW you can call sqlite3_initialize within a project that uses sqlite_orm and it will be ok.
This is all I can help you with. If you have any idea how I can help you else please let me know.

[MSonn]

It seem that we were talking past each other. I really just suggested a little section in the documentation that helps the user to figure out what happens if variant X of libsqlite3 is used with sqlite_orm. Something like: "SQLITE_OMIT_AUTOINIT: sqlite_orm does not call sqlite3_initialize. If this option is used then the user has to call sqlite3_initialize prior to calling make_storage as described in the sqlite3 documentation".

Sqlite_orm is not "just" a libsqlite3 wrapper, therefore one cannot figure this out by just reading the libsqlite3 documentation.

The added functions help too. Especially if someone cannot just compile libsqlite3 with different options. I would however suggest adding a RAII class that handles the initialization/shutdown part automatically.

[fnc12]

RAII class is a thing to discuss cause it makes something unclear: you don't know whether you need to call sqlite3_initialize by yourself or not. But when you cannot find any SQLITE_OMIT_AUTOINIT keyword in readme or wiki then it means that sqlite_orm has nothing to do with SQLITE_OMIT_AUTOINIT for now and you should use it as is on your own. This is how I should think if I use a third party lib.

About adding new API and the whole feature in sqlite_orm:
Documentation says that sqlite3_initialize is called inside every sqlite3_open call so we do not have to call it explicitly. Why do you need this? Do you use embedded system? I'd like to know more about your crash and your environment to make the best improvement to make your software development more pleasant. Thanks

[MSonn]

I am afraid that I don't understand this point. I meant a RAII class that only calls sqlite3_initialize in the ctor and sqlite3_shutdown in the dtor. It is (in my opinion) clear what it does if it also called "Initializer" or something like that, isn't it?

I agree that you shouldn't expect that something works if it is not mentioned anywhere. The main issue is that those options are recommended by the sqlite3 devs. The documentation mentions that few might not work everywhere (e.g. SQLITE_OMIT_AUTOINIT). But thats pretty much it. As those options are recommened I thought that sqlite_orm would work with those options out of the box. My suggestion is just about helping other devs that might encounter this issue in the future.

I am working in an "embedded" linux environment, however those options were just enabled, because they were recommended...
I fixed it by removing most of the OMIT options already.

[fnc12]

Of course I can add RAII class that will call sqlite3_initialize in its ctor. But then I am sure that will appear people who will want to avoid automatic calling sqlite3_initialize during static memory initialization and they will ask to remove it.
It is better to fix your problem. I don't understand what exactly did you change. You added defines for SQLITE_OMIT_AUTOINIT into your build compile time options and then everything worked well?

[MSonn]

Sqlite_orm should not instantiate this class by default e.g. when make_storage is called. This should be done by the user. It should not bother anyone this way.

I fixed the issue by removing the defines for SQLITE_OMIT_AUTOINIT once I found out what's going on. Afterwards I created this issue. Documentation regarding the libsqlite3 options might help devs that experience similar issues in the future.

[fnc12]

removing the defines for SQLITE_OMIT_AUTOINIT

What do you mean? Did you added #undef SQLITE_OMIT_AUTOINIT or removed a compiler option SQLITE_OMIT_AUTOINIT or you altered sqlite's code?

[MSonn]

I did not change the source code. I removed the define from the compiler call.

[fnc12]

Ok. The main topic here is documentation. What kind of text in the docs do you expect? sqlite_orm has nothing to do with SQLITE_OMIT_AUTOINIT cause SQLITE_OMIT_AUTOINIT affects code inside sqlite_open functions.

[MSonn]

If SQLITE_OMIT_AUTOINIT is set then the sqlite3 user has to call the initialize functions. This is in my opinion still related to sqlite_orm, because it acts as a frontend to sqlite3. A caveat section would be therefore useful:

• sqlite3_initialize must be called but sqlite_orm does not call this function (automatically)
• sqlite_orm API that can be called to handle this case (e.g. sqlite_orm::initialize) if you decide to support this use case

[fnc12]

If SQLITE_OMIT_AUTOINIT is set then the sqlite3 user has to call the initialize functions. This is in my opinion still related to sqlite_orm, because it acts as a frontend to sqlite3.

This is a part of SQLite documentation. I don't see the reason to copy SQLite docs to sqlite_orm docs.
We can add sqlite_orm::initialize if course and it must be documented. But all that docs must say is calls 'sqlite3_initialize' inside. And add link to 'sqlite3_initialize' SQLite doc.

[MSonn]

Yes this is documented there as well, however sqlite_orm is not "just" a c++ wrapper for sqlite3. It is difficult to tell what sqlite3 functions sqlite_orm actually calls from a user perspective.

[fnc12]

Actually it is just a frontend for sqlite3. Why is it so? Cause all it does can be implemented with raw sqlite3 except features that are not available in pure C - static query validations. About

It is difficult to tell what sqlite3 functions sqlite_orm actually calls from a user perspective.

you are right. The lib is not well documented so one has to look inside the code sometimes to understand what happens inside. But sqlite_orm aims to have clean API - one function call per one query. Exceptions are very rare: e.g. aliases. Anyway if sqlite_orm would have every function documented text about sqlite3_initialize would not exist cause sqlite_orm does not support it right now.

[fnc12]

is the issue actual?

[fnc12]

@MSonn are you there?

[MSonn]

Sorry for the late response. This issue can be closed.