json.update and vector<pair>does not work with ordered_json

[tawmoto]

Hi,

I am trying to convert my code from json to ordered_json, but I get some problems.

    ordered_json jsonAnimals = {
            {"animal", "dog"}};

    ordered_json jsonCat = {
            {"animal", "cat"}};

    jsonAnimals.update(jsonCat);

I get the same problem with the following code (which works with json)

    std::vector<std::pair<std::string, int64_t>> intData = {std::make_pair("aaaa", 11),
                                                            std::make_pair("bbb", 222)};

    ordered_json jsonObj;

    for (const auto& data : intData)
    {
        jsonObj[data.first] = data.second;
    }

Both generate the same error:

In instantiation of ‘void nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::update(nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::const_reference) [with ObjectType = nlohmann::ordered_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; BinaryType = std::vector<unsigned char>; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::const_reference = const nlohmann::basic_json<nlohmann::ordered_map>&; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::value_type = nlohmann::basic_json<nlohmann::ordered_map>]’:
/home/tawfic/CLionProjects/untitled2/main.cpp:575:42:   required from here
/home/tawfic/Desktop/fpc/json.hpp:22178:50: error: cannot bind rvalue reference of type ‘std::__cxx11::basic_string<char>&&’ to lvalue of type ‘const key_type’ {aka ‘const std::__cxx11::basic_string<char>’}
             m_value.object->operator[](it.key()) = it.value();
In file included from /home/tawfic/CLionProjects/untitled2/main.cpp:562:
/home/tawfic/Desktop/fpc/json.hpp:16436:8: note:   initializing argument 1 of ‘T& nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>::operator[](Key&&) [with Key = std::__cxx11::basic_string<char>; T = nlohmann::basic_json<nlohmann::ordered_map>; IgnoredLess = std::less<void>; Allocator = std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::ordered_map> > >]’
     T& operator[](Key&& key)

Should I do something special if I convert from json to ordered_json?

Thank you :+1:

[nlohmann]

I cannot reproduce this with release version 3.9.0 with Apple clang version 12.0.0 (clang-1200.0.22.19).

Which version of the library and compiler version are you using?

Example 1

#include <iostream>
#include <nlohmann/json.hpp>

using ordered_json = nlohmann::json;

int main()
{
    ordered_json jsonAnimals = {
            {"animal", "dog"}};

    ordered_json jsonCat = {
            {"animal", "cat"}};

    jsonAnimals.update(jsonCat);

    std::cout << jsonAnimals << std::endl;
}

Output:

{"animal":"cat"}

Example 2

#include <iostream>
#include <nlohmann/json.hpp>

using ordered_json = nlohmann::json;

int main()
{
    std::vector<std::pair<std::string, int64_t>> intData = {std::make_pair("aaaa", 11),
                                                            std::make_pair("bbb", 222)};

    ordered_json jsonObj;

    for (const auto& data : intData)
    {
        jsonObj[data.first] = data.second;
    }

    std::cout << jsonObj << std::endl;
}

Output

{"aaaa":11,"bbb":222}

[tawmoto]

Hi Niels, I am using gcc 8.3, Linux.
json library 3.9.0
I will share the project with you tomorrow maybe.

[nlohmann]

Hi Niels, I am using gcc 8.3, Linux.

And which library version? d34771cafc87b358ba421faca28facc7f8080174?

[tawmoto]

@nlohmann Maybe it's a problem in your example?
using ordered_json = nlohmann::json; should be using ordered_json = nlohmann::ordered_json;
This is how I am reproducing it.

So I am using the released version: https://github.com/nlohmann/json/releases/download/v3.9.0/json.hpp

[nlohmann]

Sorry, typo. Now I can reproduce the error.

There seems to be a missing operator in nlohmann::ordered_map. I'll have a look.

[nlohmann]

/cc @gatopeich

[tawmoto]

Thank you Niels. :+1:

[nlohmann]

I opened PR #2319 - I would appreciate any feedback on this!

[tawmoto]

From my first tests I get other errors, I am using json.hpp from the PR branch (md5 e41e2426084f65bf5ca5652651ffd13b)
I have to dive more in the problem, to see what is the exact line.

In instantiation of ‘IteratorType nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::erase(IteratorType) [with IteratorType = nlohmann::detail::iter_impl<nlohmann::basic_json<nlohmann::ordered_map> >; typename std::enable_if<(std::is_same<InputIT, nlohmann::detail::iter_impl<nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType> > >::value || std::is_same<InputIT, nlohmann::detail::iter_impl<const nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType> > >::value), int>::type <anonymous> = 0; ObjectType = nlohmann::ordered_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; BinaryType = std::vector<unsigned char>]’:
/home/tawfic/Desktop/fpc/json.hpp:5625:21:   required from ‘bool nlohmann::detail::json_sax_dom_callback_parser<BasicJsonType>::end_object() [with BasicJsonType = nlohmann::basic_json<nlohmann::ordered_map>]’
/home/tawfic/Desktop/fpc/json.hpp:10310:33:   required from ‘bool nlohmann::detail::parser<BasicJsonType, InputAdapterType>::sax_parse_internal(SAX*) [with SAX = nlohmann::detail::json_sax_dom_callback_parser<nlohmann::basic_json<nlohmann::ordered_map> >; BasicJsonType = nlohmann::basic_json<nlohmann::ordered_map>; InputAdapterType = nlohmann::detail::iterator_input_adapter<__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string<char> > >]’
/home/tawfic/Desktop/fpc/json.hpp:10201:13:   required from ‘void nlohmann::detail::parser<BasicJsonType, InputAdapterType>::parse(bool, BasicJsonType&) [with BasicJsonType = nlohmann::basic_json<nlohmann::ordered_map>; InputAdapterType = nlohmann::detail::iterator_input_adapter<__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string<char> > >]’
/home/tawfic/Desktop/fpc/json.hpp:23073:9:   required from ‘static nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType> nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::parse(InputType&&, nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::parser_callback_t, bool, bool) [with InputType = const std::__cxx11::basic_string<char>&; ObjectType = nlohmann::ordered_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; BinaryType = std::vector<unsigned char>; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::parser_callback_t = std::function<bool(int, nlohmann::detail::parse_event_t, nlohmann::basic_json<nlohmann::ordered_map>&)>]’
/home/tawfic/work/genesis/fpcsdk/protocol/src/json_formatter.cpp:142:44:   required from here
/home/tawfic/Desktop/fpc/json.hpp:20493:45: error: no matching function for call to ‘nlohmann::ordered_map<std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::ordered_map>, std::less<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::ordered_map> > > >::erase(std::vector<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::ordered_map> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::ordered_map> > > >::iterator&)’
                 result.m_it.object_iterator = m_value.object->erase(pos.m_it.object_iterator);
In file included from /home/tawfic/work/genesis/fpcsdk/protocol/src/json_formatter.cpp:5:
/home/tawfic/Desktop/fpc/json.hpp:16472:15: note: candidate: ‘typename nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>::Container::size_type nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>::erase(const Key&) [with Key = std::__cxx11::basic_string<char>; T = nlohmann::basic_json<nlohmann::ordered_map>; IgnoredLess = std::less<std::__cxx11::basic_string<char> >; Allocator = std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::ordered_map> > >; typename nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>::Container::size_type = long unsigned int]’
     size_type erase(const Key& key)

[nlohmann]

In the same code?

[tawmoto]

it's my production code, yes.
It seems that the error is from the parsing now ordered_json jsonObj = json::parse(jsonResponse);

I guess we are not allowed to use ordered_json here?

[nlohmann]

No, you should be able to use ordered_json::parse.

[tawmoto]

I think a good test for this new feature (ordered_json) will be to re-run all the tests with ordered_json instead of json :-)

[nlohmann]

Yes, indeed. We have not covered the full interface of std::map so far which is biting us now. I will update the regression test in #2319 and see that I can fix it.

[tawmoto]

Thank you, for me is not urgent, I will keep using json object.

[nlohmann]

Can you please check again with f13af83a9499dd26998f482111ca3c654a231279 ?

[tawmoto]

Yes, it works with the latest update, great!

Btw, is there a task "Run all the tests with ordered_json", so I can subscribe to it?
I would like to change from json to ordered_json in all our projects after everything is tested.

Thanks a lot again, Niels!

[nlohmann]

I don't have an issue for executing all tests for ordered_json as well. Given the large amount of tests, this would be unfeasible. What I started to do instead is to make sure ordered_map implements the same interface as std::map does which is the default type for JSON objects. Then I can make sure the future won't introduce compilation errors as it did in your case.

Right now, #2319 implements all calls to std::map functions I could find in the code. I will work to complete them. However, I am currently facing a nasty MinGW bug which I will need some time to circumvent:

Fatal error: can't write 310 bytes to section .text of test/CMakeFiles/test-regression.dir/src/unit-regression.cpp.obj: 'File too big'

FYI: If you want to support me and my work on this project, you can become a sponsor on GitHub, or used Paypal.