Json: type conversion failing with clang ext_vector_type

Created on 14 Oct 2020 · 7Comments · Source: nlohmann/json

Hi, I would like to serialize a vector type defined using clang's ext_vector_type extension,
but this results in the errors reported below.
I am posting this as a bug, but I am not sure there is a viable workaround, so hope it is ok.
Thanks in advance,
Daniel

What is the issue you have?

I specify the to_json and from_json functions for a vector type defined as

typedef float v2 __attribute__((ext_vector_type(2)));

But the type is not recognized by the library and this results in compilation errors.
Explicitly calling to_json and from_json works.
Defining the type as a struct also works as expected.

Please describe the steps to reproduce the issue.

Here is a minimal example that reproduces the error. I commented different lines that also produce similar errors.

#include "json.hpp"

using json = nlohmann::json;

typedef float v2 __attribute__((ext_vector_type(2)));
// struct v2{ float x; float y; }; // using a struct instead of the above, all the code below works as expected

void to_json(json& j, const v2& x) {
  std::vector<double> y;
  y.push_back(x.x);
  y.push_back(x.y);
  j = y;
}

void from_json(const json& j, v2& x) {
  std::vector<double> y;
  auto                it = j.begin();
  x.x                    = *it;
  it++;
  x.y = *it;
}

int main(int, char**) {
  json j;
  v2   foo;
  foo = j.at("foo").get<v2>();
  // the following also produce compilation errors:
  // foo = j['foo'];
  // j.get_to<v2>(foo);
  // j["foo"] = foo;  //  Also reverse does not work

  // This does work:
  // from_json(j, foo);
  return 0;
}

Can you provide a small but working code example?

See above.

What is the expected behavior?

I would expect the definitions of to_json and from_json to result in a successful compilation of the code.

And what is the actual behavior instead?

Using ext_vector_type results in the following errors:

main.cpp:27:21: No matching member function for call to 'get'

json.hpp:19419:16: Candidate template ignored: requirement 'std::is_same<float __attribute__((ext_vector_type(2))), nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long long, unsigned long long, double, std::allocator, adl_serializer, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > > >::value' was not satisfied [with BasicJsonType = float __attribute__((ext_vector_type(2)))]
json.hpp:19442:19: Candidate template ignored: requirement 'detail::is_basic_json<float __attribute__((ext_vector_type(2)))>::value' was not satisfied [with BasicJsonType = float __attribute__((ext_vector_type(2)))]

json.hpp:19492:15: Candidate template ignored: requirement 'detail::has_from_json<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long long, unsigned long long, double, std::allocator, adl_serializer, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > >, float __attribute__((ext_vector_type(2))), void>::value' was not satisfied [with ValueTypeCV = float __attribute__((ext_vector_type(2))), ValueType = float __attribute__((ext_vector_type(2)))]

json.hpp:19543:15: Candidate template ignored: requirement 'detail::has_non_default_from_json<nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long long, unsigned long long, double, std::allocator, adl_serializer, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > >, float __attribute__((ext_vector_type(2))), void>::value' was not satisfied [with ValueTypeCV = float __attribute__((ext_vector_type(2))), ValueType = float __attribute__((ext_vector_type(2)))]

json.hpp:19698:10: Candidate template ignored: substitution failure [with PointerType = float __attribute__((ext_vector_type(2)))]: no matching member function for call to 'get_ptr'

json.hpp:19710:20: Candidate template ignored: substitution failure [with PointerType = float __attribute__((ext_vector_type(2)))]: no matching member function for call to 'get_ptr'

The reverse procedure (converting to json) also produces this error:

json.hpp:18745:17: Candidate function not viable: no known conversion from 'v2' (vector of 2 'float' values) to 'nlohmann::basic_json<std::map, std::vector, std::__1::basic_string<char>, bool, long long, unsigned long long, double, std::allocator, adl_serializer, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > >' for 1st argument

Which compiler and operating system are you using?

I am using Apple clang version 11.0.0, on OSX Mojave.

Which version of the library did you use?

[ ] latest release version 3.9.1
[ ] other release - please state the version: ___
[x] the develop branch

If you experience a compilation error: can you compile and run the unit tests?

[x ] yes
[ ] no - please copy/paste the error message below

confirmed bug proposed fix

Source

colormotor

All 7 comments

I can confirm your observations, but I have no idea why it does not work or how to fix it.

nlohmann on 14 Oct 2020

👍1

Thanks for the quick reply!
Not ideal, but for the moment I use a wrapper with a constructor and cast operator for the cases where I really need to serialize this type. Will keep you posted if I find a better solution.

colormotor on 14 Oct 2020

Hello
you may have an issue with your #include directives. Can you check the order of your files and if you declared the class in the file of a friend method.

diabitbol on 9 Dec 2020

Hey @colormotor I am still digging into this, now that I am finally on holidays. I think it has to do with argument-dependent lookup and this design that the library uses. I think I managed to simplify a bit what is happening in the library in this compile explorer link: https://godbolt.org/z/xrY469 please play around with it I will try to ask someone who is a compiler guru. The error reads something like

<source>:25:12: error: call to function 'foo' that is neither visible in the template definition nor found by argument-dependent lookup
    return foo(alice, bob);

As a better workaround is to encapsulate the {from/to}_json functions into the nlohmann namespace. It seems to fix the issue link:

#include "nlohmann/json.hpp"

using json = nlohmann::json;

typedef float v2 __attribute__((__ext_vector_type__(2)));

namespace nlohmann {
void from_json(const json& j, v2& x) {
  // Fill in as you please
}
void to_json(json& j, const v2& x) {
  // Fill in as you please
}
}

int main(int, char**) {
  json j;
  v2   foo;
  foo = j.get<v2>();
  j =  foo;
  return 0;
}

I also don't think it has to do with the __attribute__ fields. If you just specify typedef float v2 you get the same error, please reference the first compiler explorer link I provided. This explains why you are the first one to discover this issue since 99% of the usage of this function if with user defined structs/classes and not with modified built in types since the library already provides defaults for those cases.

I will keep digging and I will keep you posted