Solidity: Consider replacing jsoncpp

[ekpyron]

• The repo looks like jsoncpp only receives minimal maintenance.
• The workaround of manually downloading jsoncpp in cmake is evil (we could of course attempt to change that in isolation, but there don't seem to be package updates for jsoncpp e.g. in recent ubuntu either making this a continuing hassle).
• jsoncpp is neither the fastest nor the smallest nor the most conformant option.
• There are a number of pretty much drop-in replacements available.
• We can easily move to a header-only library which pretty much avoids all dependency issues in perpetuity.

I'd suggest to consider replacing jsoncpp with either http://rapidjson.org/ or https://github.com/nlohmann/json (or any other of the available candidates).

Is there any reason for staying with jsoncpp other than the required work a move would be (which I wouldn't expect to actually be too bad)?

Closes #3557.

[christianparpart]

wrt. nlohmann/json, they state Our whole code consists of a single header file json.hpp, which would be the main reason for me to choose them.

Also the examples he provides in the readme look very convincing to me. => :+1: for this one.

[chriseth]

I had a quick look over nlohmann and found the following two items that might be issues:

• https://github.com/nlohmann/json#comments-in-json (just remember we had some discussion about that at some point)
• https://github.com/nlohmann/json#order-of-object-keys - we have to check if that can lead to nondeterminism

[ekpyron]

I think while the insertion order of the keys is not preserved, they don't have an "undefined" order - they use a std::map, so in the end the keys will be ordered by string comparison and that's deterministic - but we need to recheck, yes.

Regarding comments I'm not sure - should we allow them even though they are actually not valid json? I know that json-cpp only supports them in a weird way as well (whether a comment is fine depends on whether it's key-comment-comma or key-comma-comment, if I recall correctly).

[nlohmann]

Hi, I'm the author for nlohmann/json. Please let me know if you need any assistance!

[chriseth]

@nlohmann great to have you here! Can you comment on the order of keys? Is it a defined order that may be different from the insertion order or can it depend on things like memory layout?

[nlohmann]

We use an std::map internally, so the order when iterating over an object is sorted alphabetically w.r.t. std::string::operator<.

[chriseth]

Wonderful, thanks!

[aarlt]

Ah nice, I just found this issue. @ekpyron I had the same impression and I think it would be nice to replace jsoncpp. Around a year ago I created a branch to replace jsoncpp but somehow I never created a PR (https://github.com/aarlt/solidity/tree/nlohmann-json). I don't remember exactly what the state was, but if you would like to replace jsoncpp with nlohmann/json, I could support you.

[nlohmann]

As I wrote earlier, just let me know if you need assistance!

[ekpyron]

I brought this up in our weekly call earlier and @chriseth said we should postpone this until after 0.6.0 is released.
However https://github.com/aarlt/solidity/commit/1329c25756f2b242d9af7860eda6f37a72d38dc0 is a great reference for estimating how much effort this will be - it doesn't look too bad, so I think we can do this quickly after 0.6.0.

[axic]

nlohmann's seems to be better in every single benchmark compared to jsoncpp: https://github.com/miloyip/nativejson-benchmark

[chriseth]

Are benchmarks the benchmark? I would say it is more important that the code is easy to understand and does not create any ambiguities.

[axic]

I think the API of nlohman was quite good, even compared to jsoncpp.

The benchmark categories are:

• code size
• parsing speed
• serialisation speed

There is a huge difference between speeds, which can be an issue with large compilations (large JSON outputs or input). I think this may be the reason people were complaining on compilation speed.

[ekpyron]

By the way: @Marenz is having problems problems with the performance of the AST export and import tests, for which I think the json parsing and emitting part is a major bottleneck - so it's not like performance is entirely irrelevant... and regarding readability: just have a quick look at https://github.com/aarlt/solidity/commit/1329c25756f2b242d9af7860eda6f37a72d38dc0 - the API is very similar to jsoncpp and closer to STL, which I'd consider an advantage.

[axic]

Regarding comments I'm not sure - should we allow them even though they are actually not valid json? I know that json-cpp only supports them in a weird way as well (whether a comment is fine depends on whether it's key-comment-comma or key-comma-comment, if I recall correctly).

I would say there is no point supporting comments when they are not a valid JSON feature. IIRC we have them disabled as much as jsoncpp allows disabling them.

Also apparently nlohman-json now supports CBOR integration: https://github.com/nlohmann/json#binary-formats-bson-cbor-messagepack-and-ubjson

This means we could in theory replace our code or at least replace it in the tests. However this is not a pressing issue as our small CBOR code works well and as intended.