Json: The compilation time is slow

[certik]

I used nlohmann::json in some AST tree to JSON conversion in a compiler, and it was very easy to get started, documentation is good, syntax is simple and it just works. However, I noticed that the compilation time of my code got really slow. So I switched to rapidjson, it took me a lot longer to get started, but when all was done, the compilation of one file got from 4.8s to 1.8s, here is the patch that achieves this speed up:

https://gitlab.com/lfortran/lfortran/commit/904cb6301e86f95e6ad86d571ea74b23cd2b0e70

and another file from 5.7s to 2.8s:

https://gitlab.com/lfortran/lfortran/commit/bfe179ec2ab72855d273db1258c858857ddb9fa2

I am posting it here as a feedback, if there is anything that you can do to speedup the compilation, that would be very helpful.

[volanin]

I just want to +1 this suggestion.

Compilation with nlohmann::json is really, really slow. I use it extensively in my company's project, but since we're in the initial phase of rapid prototyping, the high compilation times are making development grind to a halt.

I found this ticket while searching for a way to improve compilation times with nlohmann::json. Even gcc precompiled headers aren't solving the problem.

[pmbittner]

Something that could help is the forward declaration header. I used it in my project so that I only had to include json in my cpp files.
However, the json_fwd.hpp does not work ad-hoc with the singleinclude json header (you have to remove the default template arguments in json.hpp).

Another suggestion would be to introduce variability, maybe through ifdefs or multiple instead of a single header file. There are lots of cool features in nlohmann::json but as I do not need all of them, turning some things off for compilation would help, too.

[volanin]

I don't think json_fwd.hpp works together with NLOHMANN_JSON_SERIALIZE_ENUM. The enums serialization must be defined in the header file, and it requires json.hpp. Or maybe I'm doing something wrong...

[ClaudiuHKS]

Constexpr slow down compilation time but should help execute code faster.

I am not REALLY sure of what I said above but I hide strings in some of my binaries using Constexpr and I see a huge difference while compiling .. like it takes more time to finish when using Constexpr.

[nlohmann]

@ClaudiuHKS Do you have some benchmarks on the effect of constexpr?

[ClaudiuHKS]

@ClaudiuHKS Do you have some benchmarks on the effect of constexpr?

With Constexpr

include

void main (void) { constexpr char Data [8192] = { '-', }; printf ("%s\n", Data); };

Without Constexpr

include

void main (void) { char Data [8192] = { '-', }; printf ("%s\n", Data); };

You should really see the difference yourself, while compiling them, like it takes much more time to compile when Constexpr exists, but I think the code will execute faster when it runs, rather than when Constexpr is absent.

NOTE! I have tested it with Visual Studio 2019 on Windows 10, I have no idea if it's the same for other operating systems.

CPPReference states that _the constexpr specifier declares that it is possible to evaluate the value of the function or variable at compile time_ so IMO affecting code at compile time automatically changes code at run time, at least, the way it's being executed.

[nlohmann]

I cannot reproduce this on macOS:

[ClaudiuHKS]

https://pastebin.com/Wbz0rHd6

With Constexpr = Time Elapsed 00:00:02.57
Without Constexpr = Time Elapsed 00:00:00.36

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.