Json: [Question] How do I parse JSON into custom types?

[rcdailey]

Reviewing the documentation, I don't see a lot of helpful examples on more complex parsing scenarios. Really it seems like most of the documentation describes going from types to JSON, but I need to completely deserialize JSON data into C++ objects.

Here is the JSON data I'm working with:

{
  "campaigns": [{
      "name": "patt_coupon",
      "start_date": "2019-01-01",
      "end_date": "2019-12-31",
      "push": "pass.com.company.coupon.5dollar",
      "pull": [
        "pass.com.company.coupon",
        "pass.com.company.coupon_revised"
      ]
    },
    {
      "name": "loyalty",
      "start_date": "2019-01-01",
      "end_date": "2019-12-31",
      "push": "pass.com.company.loyalty.program1",
      "pull": [
        "pass.com.company.loyalty.program1"
      ]
    }
  ]
}

For the campaigns array, I want to parse each element into an array of custom objects:

#include <fstream>
#include <stdexcept>
#include <boost/date_time/gregorian/gregorian.hpp>
#include <fmt/format.h>
#include <nlohmann/json.hpp>

using json = nlohmann::json;
namespace boost
{
   namespace gregorian
   {
      void from_json(json& j, boost::gregorian::date& d)
      {
         std::string dateValue;
         j.get_to(dateValue);
         d = boost::gregorian::from_string(dateValue);
      }
   }
}

class WalletCampaignConfiguration::WalletCampaign
{
public:
   WalletCampaign(json& config)
   {
      config.at("start_date").get_to(startDate);
   }

   boost::gregorian::date startDate, endDate;
   std::string pushPass;
   std::vector<std::string> pullPasses;
};

int main() {
   std::ifstream file{jsonPath}; // this points to the file containing the JSON above
   if (!file)
   {
      throw std::runtime_error{"Unable to open wallet config json: " + jsonPath.string()};
   }

   json config;
   file >> config;

   for (auto const& campaignJson : config.at("campaigns"))
   {
      m_campaigns.push_back(std::make_unique<WalletCampaign>(campaignJson));
   }

   auto const& c = *m_campaigns.at(0);
   std::cout << "start: " << c.startDate << "\n\n";
   std::cout << "end: " << c.endDate << "\n\n";
   std::cout << "push: " << c.pushPass << "\n\n";

   for (auto const& p : c.pullPasses)
   {
      std::cout << "pull: " << p << "\n\n";
   }
}

I'm compiling this using Visual Studio 2019 with C++17 enabled. The code above doesn't compile, with these errors:

1>E:\code\frontend\source\Core\Wallet\Source\Wallet\WalletCampaignConfiguration.cpp(47,30): error C2672:  'nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get_to': no matching overloaded function found
1>E:\code\frontend\source\Core\Wallet\Source\Wallet\WalletCampaignConfiguration.cpp(47,1): error C2783:  'ValueType &nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get_to(ValueType &) noexcept(<expr>) const': could not deduce template argument for '__formal'
1>E:\code\frontend\source\Core\External\json\include\nlohmann/json.hpp(2680): message :  see declaration of 'nlohmann::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::get_to'

I'm not sure why it doesn't compile. But I'm sure it has to do with something about how I'm trying to use the library. Can you give me an example of how to parse this JSON document in an object oriented way as I'm trying to do?

[nlohmann]

I don't have time to compile the code in the moment. But one note: The signature of the from_json function must be

void from_json(const json& j, boost::gregorian::date& d)

instead of

void from_json(json& j, boost::gregorian::date& d)

(see https://github.com/nlohmann/json#basic-usage).

[rcdailey]

Thanks for the quick response. To help you out, I put this code on Wandbox along with your single header file and the above JSON. You can easily compile it that way:

https://wandbox.org/permlink/rgECm599lctiFXkA

Note with your fix it still has problems. The for loop where I construct the unique_ptr to WalletCampaign doesn't work. I guess the elements I'm getting aren't json types. A big part of this is intended to be a code review. If you have the time, I want to know the idiomatic way of doing what I'm trying to do. I feel like I'm doing this the hard way. Your library seems like it should simplify a lot of my boilerplate.

Again I appreciate your time. Let me know if you get a spare moment to code review this. Thanks!

[nlohmann]

Your constructor should accept a const reference:

WalletCampaign(const json& config)

That makes the code compile at Wandbox.

You may want to replace the constructor taking a json to a from_json function, but this is just style.

What makes you feel you write boilerplate code?

[rcdailey]

Thanks, your fix was the missing piece. The compiler diagnostic did not make the const issue obvious. Very cryptic output. Maybe you can add a static_assert to detect these issues for easier learning experience? Just a thought...

In terms of boilerplate, I'm still learning things, but so far making JSON elements "optional" is pretty tedious. There's no 1 line solution to this that I've seen so far. It would be nice to see built-in support for std::optional<T>. I have seen some examples where you implement a specialization of adl_serializer, but this should be built-in IMHO. And also adl_serializer can't be specialized with just from_json (I only do one-way conversion, I don't support saving to JSON, only reading from JSON).

Right now I do this:

      boost::gregorian::date startDate;
      auto element = config.find("start_date");
      if (element != config.end())
      {
         element->get_to(startDate);
      }

Would be nicer instead to have something like this:

std::optional<boost::gregorian::date> startDate;
config.try_get_to("start_date", startDate);

at() will throw so I can't use that.

EDIT: Actually my example of find() above doesn't even compile... diagnostic output again is very difficult to comprehend. Doesn't explain the problem.

EDIT2: Looks like it doesn't compile because I was wrapping the date object with std::optional<>. If I remove optional it works.

[rcdailey]

Any objection to making these part of your library?

namespace nlohmann
{
   template<typename T>
   struct adl_serializer<std::optional<T>>
   {
      static void to_json(json& j, const std::optional<T>& value)
      {
         if (!value)
         {
            j = nullptr;
         }
         else
         {
            // this will call adl_serializer::to_json which will
            // find the free function to_json in T's namespace!
            j = *value; 
         }
      }

      static void from_json(json const& j, std::optional<T>& value)
      {
         if (j.is_null())
         {
            value.reset();
         }
         else
         {
            // same as above, but with adl_serializer<T>::from_json
            value = j.get<T>();
         }
      }
   };

   template<typename T>
   struct adl_serializer<std::unique_ptr<T>>
   {
      static void to_json(json& j, const std::unique_ptr<T>& value)
      {
         if (!value)
         {
            j = nullptr;
         }
         else
         {
            j = *value;
         }
      }

      static void from_json(json const& j, std::unique_ptr<T>& value)
      {
         if (j.is_null())
         {
            value.reset();
         }
         else
         {
            value = std::make_unique<T>();
            *value = j.get<T>();
         }
      }
   };
}

Probably need to add one for shared_ptr too.

[nlohmann]

(There is a function contains to check if a key is present in an object. This may clean up code in case you don't want to compare iterators.)

I'm hesitating to add the conversions for optionals and smart pointers right now. I need to understand what this may break first.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.