Crystal: JSON objects output invalid JSON on to_s

[Serdnad]

Not a bug per se, but it seems like a questionable design decision IMO to have JSON objects print like a hash instead of as a string of valid JSON on conversion to a string.

For example,

json = JSON.parse("{\"name\": \"David\"}")
json.to_s
# output: "{\"name\" => \"David\"}"

If it weren't for the use of an arrow => instead of a colon :, the above would be valid JSON, and I could rely on implicit conversion, as opposed to being required to use #.to_json on a JSON object. In the case that I don't have access to the JSON object or, more likely, that it's inconvenient to handle JSON objects separately, it's necessary to use json.to_s.gsub("=>", ":") everywhere.

I'm wondering if this is intentional and it's been discussed, or if it simply stems from a minor implementation detail. At any rate, is this is something that could be considered for a change? I would imagine not many people if any are currently relying on the current format, and any (like myself) that are already converting to valid JSON through a substitution would be unaffected.

As a side note, however, the use of an arrow here seems perfectly fine:

json = JSON.parse("{\"name\": \"David\"}")
json
# output: {"name" => "David"}

because it's simply describing the object. Of course, if any change is made, it wouldn't hurt to apply the change here for consistency as well.

[Sija]

The method intended for JSON serialization is #to_json, not #to_s, see docs for more detailed explanation.

[Blacksmoke16]

You're getting a String representation of JSON::Any, which is represented as a Hash. If you want the actual JSON string you should use .to_json.

[Serdnad]

@Sija I understand that when I'm converting from a hash or an array, but I don't see why an actual JSON object can't just treat this as default behavior.

[Blacksmoke16]

You're not converting from a Hash or Array. You're parsing a JSON string into another data type. I'm not sure I see the problem?

[Blacksmoke16]

Actually I think I see your point. Is there a reason to _not_ just do .to_json as the default to_s behavior? Seems like that would make sense...

[Serdnad]

Yeah, I think my mention of a Hash or Array was beside the point, and I hadn't really considered that to_json really just is parsing an object to a String. But if to_s is just a string representation of an object, it seems convenient (not to mention logical imo) that the string representation of a JSON object be valid JSON itself.

[straight-shoota]

I guess it would make some sense for JSON::Any#to_s to actually show a JSON representation, i.e. delegate to raw#to_json instead of raw.to_s.

But JSON::Any is intended as a kind of thin layer around the raw value it wraps and can even be used interchangebly in some situations (comparison, hashing). I'm not sure that's the best idea, but long as it works like that, it should have the same string representation as the wrapped raw value.

This also depends on the data type of the wrapped value. The difference between #to_s and #to_json on a Hash is very small. On Array Array and most scalar types it's usually pretty much the same. But on a String, it makes a huge difference because #to_json quotes the value. So #to_json is similar to String#inspect, not String#to_s.

I think the current behaviour is fine. You just need to use #to_json when you want a JSON representation. That's expected and documented.

[jhass]

Yes, I think the misconception here is that there's any such thing as a "JSON Object type" in Crystal, which there is not.

[Serdnad]

I see, that's a fair argument, although I do feel like the difference between a proper JSON object type and a raw value in a "JSON wrapper" seems pretty small.

With that said, I've looked a little more closely into my use case, and, quite awkwardly, it seems like my issue isn't with JSON::Any#to_s at all... 🤦 . Instead, it's with the returned value itself, i.e. the result of evaluating json at the end of my initial examples.

I have a method which accepts a value of several possible types, specifically the union of all primitives, arrays consisting of a primitive, and, now, JSON::Any. At some point, this method relies on the value passed in conjunction with the value's type. Previously, this worked because evaluating an Int results in an Int, as does evaluating a Float (result in a Float), and so on for strings and even arrays. In fact, for all cases but a hash, evaluating a value wrapped in JSON::Any will result in JSON.

However, when I evaluate an instance of JSON::Any that wraps a hash, the output is not JSON, solely because of the use of =>. Whether or not it was coincidental in the first place, it's inconsistent with all other cases, and it feels odd to me that JSON.parse(object) would output valid JSON for everything but a Hash object, simply because of an underlying implementation detail.

Also, I apologize for the abrupt turn in topic... Please let me know if this warrants a separate issue, a renaming, or anything of the like.

[Serdnad]

On second thought, without looking at the actual source for how JSON::Any wraps values, I would guess that on evaluation, a JSON::Any instance simply returns its wrapped value... in which case I imagine changing this behavior would be less trivial. Under that understanding, I can see that the convenience of guaranteeing that JSON.parse(object) output valid JSON might not be worth the work...

Edit: Yeah, I think I might've confused myself trying to hack some functionality into some existing libraries. Other than keeping this open for the initial request of having to_s use to_json, I think this can be closed. Thanks for all the explanations, I've learned quite a bit.

[jhass]

I'm not sure I can quite follow you. Do you have some minimal self contained example to illustrate what you're talking about?

[Serdnad]

Hmm, unfortunately not at the moment, see my edit above. I'm going to try a different approach to my issue, thanks for the help.

[asterite]

When you say evaluate, what do you mean? A thing doesn't evaluate. Are you using Crystal play to try this out? Putting json evaluates to something, and json.to_s evaluates to something else doesn't make sense. json doesn't evaluate to anything, unless you are using irc or crystal play which do evaluate the objects by calling some method, presumably inspect.

So could you explain where are you seeing this behavior?

[Serdnad]

Yeah @asterite , I've been using icr to test these out. I see, I'll have to read up on what inspect does, then.

Would you mind explaining what you mean by saying things aren't evaluated though? I've been learning Haskell recently, so maybe that's cause for some of my flawed thinking

[Serdnad]

To try and clear up some of the confusion I've caused, my motivation for this is trying to make a quick and dirty patch to the DB library so that it works with JSON types, which the underlying driver library for Postgres, pg, supports. I've been able to get as far as having the library pass my wrapped value to pg, but when the value is inspected(?) right before it's passed to my Postgres database, it results in a string of invalid JSON, which chokes Postgres.

[asterite]

Could you share some code? If you need to pass a JSON string to postgres, you would call to_json on the object. But of course, without code we don't know what you are trying to do.

[jhass]

If you need to pass a JSON string to postgres, you would call to_json on the object.

Only if you want to store it as a string! PostgreSQL has a JSON datatype: https://github.com/will/crystal-pg/blob/cafee448dc09ef487bd3941febce8796830f1bd7/src/pg/decoder.cr#L379-L405

https://www.postgresql.org/docs/current/datatype-json.html
https://www.postgresql.org/docs/current/functions-json.html

[Serdnad]

Yep! That's exactly it, I'm trying to take advantage of PostgreSQL's JSON type, but when I pass in a hash value wrapped in JSON::Any, at some point along the line something happens so that the final parameter passed to the database (which I'm using put statements to check, in case that affects things) is something like {"a" => 3} instead of JSON.

I don't have one now, but I'll try to create as simple a self-contained example as I can and update this comment shortly.

[asterite]

Maybe report a bug in crystal-pg? It might be a bug of that shard.

[jhass]

is something like {"a" => 3} instead of JSON.

Make sure to use p, p!, pp, pp! for debug printing. Using puts this could be both, {"a" => 3} or "{\"a\" => 3)"!

[Serdnad]

Hmm, I'm not sure it is a crystal-pg thing. I'm currently using the Granite ORM library, and it's through that that I'm using DB and (through that) crystal-pg. To try and patch in support for my use case, I've been making tweaks all around - also the reason I haven't provided actual code.

At this moment, I think what I'm most confident about is that changing the default behavior of JSON::Any#inspect (or is it #to_s? still a little confused here) would fix my use case. Since it seemed to me like a reasonable default to have #to_s and #to_json for JSON::Any to be equivalent anyway, I opened this issue.

@jhass Oh interesting, I wasn't aware of that. Thanks, that seems really good to know.

Update: I think I might have solved my specific problem using Granite converters, not sure how I overlooked them before. If the consensus is that the current behavior of JSON::Any#to_s and #inspect is the desired one, then I'm happy to close this issue. Thanks again for all the help.