Crystal: How to distinguish absence of key-value pairs vs null values in JSON.mapping?

[akzhan]

For now I do this using:

require "json"

class Object
  def on_presence(&block)
    unless is_a?(ValueAbsence)
      yield self.not_nil!
    end
  end
end

class ValueAbsence
  Absence = ValueAbsence.new

  def initialize(pull = nil)
    raise "oops" if pull # should never be pulled by JSON parser
  end

  # Writes `"absence"` to the given `IO`.
  def inspect(io)
    io << "absence"
  end

  def self.absence
    Absence
  end
end

class UpdateUser
  JSON.mapping(
    id: {type: Int32 | ValueAbsence | Nil, nilable: true, default: ValueAbsence.absence},
    first_name: {type: String | ValueAbsence | Nil, nilable: true, default: ValueAbsence.absence},
    last_name: {type: String | ValueAbsence | Nil, nilable: true, default: ValueAbsence.absence},
    birth_date: {type: Int64 | ValueAbsence | Nil, nilable: true, default: ValueAbsence.absence},
    gender: {type: String | ValueAbsence | Nil, nilable: true, default: ValueAbsence.absence},
    email: {type: String | ValueAbsence | Nil, nilable: true, default: ValueAbsence.absence}
  )
end

p UpdateUser.from_json(%q|{"id":15, "email": "[email protected]", "gender": null}|)

https://carc.in/#/r/2jpo

somewhere in User class

class User
  def assign(update_user) : Nil
    update_user.id.on_presence do |i|
      bad_request! if i != id
    end
    update_user.first_name.on_presence do |fn|
      self.first_name = fn
    end
    update_user.last_name.on_presence do |ln|
      self.last_name = ln
    end
    update_user.birth_date.on_presence do |bd|
      self.birth_date = bd
    end
    update_user.gender.on_presence do |g|
      if g != "m" && g != "f"
        bad_request!
      end
      self.gender = g
    end
    update_user.email.on_presence do |e|
      self.email = e.not_nil!
    end
  end
end

But looks like ValueAbsence should be someway in stdlib.

class Object
  def on_presence(&block)
    yield self.not_nil!
  end

  def on_absence(&block)
  end

end

class ValueAbsence
  private Absence = ValueAbsence.new

  def initialize(pull = nil)
    raise "oops" if pull # should never be pulled by JSON parser
  end

  # Returns `true`: `ValueAbsence` has only one singleton value: `ValueAbsence.absence`.
  def ==(other : ValueAbsence)
    true
  end

  # Returns `true`: `ValueAbsence` has only one singleton value: `ValueAbsence.absence`.
  def same?(other : ValueAbsence)
    true
  end

  # Returns `false`.
  def same?(other : Reference)
    false
  end

  # Returns `0`.
  def hash
    0
  end

  # Returns an empty string.
  def to_s
    ""
  end

  # Doesn't write anything to the given `IO`.
  def to_s(io : IO)
    # Nothing to do
  end

  # Writes `"absence"` to the given `IO`.
  def inspect(io)
    io << "absence"
  end

  def self.absence
    Absence
  end

  def on_presence(&block)
  end

  def on_absence(&block)
    yield self
  end
end

[straight-shoota]

Could you describe a usecase for this differentiation?
And please write ABSENCE because Absence looks like a type name.

[Sija]

Also, IMHO there's some logical mistake here, since ValueAbsence is initialized just once and then passed to the JSON.mapping. It seems to me that you'd expect to have it initialized on every usage by JSON.mapping which is not the case.

[akzhan]

@straight-shoota

Typical use case:

PUT /users/5 should respond with 400 Bad Request when received "{first_name": null} due to external API restriction.

Absence of first_name and other keys should be allowed.

It is required by external API test tool, for example.

@Sija

Absence value exists because it's logically looks like singleton (just like nil to Nil).

[straight-shoota]

Sorry, I have no doubts about the difference in the JSON representation, I'd rather like to see how UpdateUser would use the ValueAbsence type. This is basically an internal of the JSON format and can't be directly translated into Crystal (an ivar may be nil, but it is always present). Therefore I'm skeptical about having the JSON-specific absence of a property represented as object state in Crystal. I think it would most times better be handled in the implementation of the parser (which would probably mean to write it manually). The parser could throw an exception (if appropriate) or act in a different way.

[akzhan]

Absence of key in json object is typical use case. It is already handled by parser by setting default (or nil) value. So it's OK to distinguish absence vs null.

By the way, now I have rewritten example more Crystal idiomatic way, thanks to @Sija and @straight-shoota

[straight-shoota]

You see, in Crystal you wouldn't use #on_presence but #nil?. It's the task of the JSON parser/emitter to translate that to/from JSON representaiton.
You're trying to add the concept of absent instance variables to Crystal classes which is a wrong path imho.

[akzhan]

What is your proposal? Hand made parser/handler 🚲 also will need a way to distinguish between these cases.

[akzhan]

Looks like great proposal, @asterite. Also for YAML parser too.

[RX14]

I don't like this. It seems like JSON.mapping is too bloated already. I can't see any common usecase for this. If you're writing a "validation tool" as you say then why not use PullParser directly. It'll give you much more info about exactly how the JSON is structured (order of keys, etc) and no modification of JSON.mapping is required.

[akzhan]

No. I'm writing API server that should satisfy API itself. API requires to respond with "400 Bad request" for bad value types including null but allows to have absent value.

It's very common case for JSON PUT requests (update some fields of entity)

And API client has corresponding load/validation/performance tool (based on yandex-tank). I must satisfy its requirements (400, 404 responses etc.).

[straight-shoota]

@RX14 Perhaps it would be an idea to further modularize JSON.mapping (similar to #4772).
But this particular case is probably generic enough (especially considering the PUT usecase) that it would make sense to have this implemented in a way like @asterite suggested.

[faustinoaq]

Hey, Do you think this question is good for stackoverflow?

[ysbaddaden]

futher modularize JSON.mapping

I totally support @RX14: JSON.mapping is meant as a simple tool to quickly map a JSON schema to a Crystal struct or class and is easy enough for 80 percentile (or hacking). The other 20 percentile use cases would be better off using PullParser directly. It's easy enough to deal with (see https://github.com/crystal-lang/shards/blob/master/src/spec.cr#L72).

[straight-shoota]

@ysbaddaden When you directly use PullParser you wouldn't have the other features of JSON.mapping. If they are modularized and can be used individually, that would be nice I think. My suggestion was not about to change the behaviour of JSON.mapping or make it more complex, but to make it possible to use only parts of it's functionality like JSON.def_to_json from #4772 does.

[akzhan]

Just finished specs for asterite proposal, will open PR after arrival to
home (a ~hour)
чт, 17 авг. 2017 г. в 20:06, Johannes Müller [e-mail]:

@ysbaddaden https://github.com/ysbaddaden When you directly use
PullParser you wouldn't have the other features of JSON.mapping. If they
are modularized and can be used individually, that would be nice I think.
My suggestion was not about to change the behaviour of JSON.mapping or
make it more complex, but to make it possible to use only parts of it's
functionality like JSON.def_to_json from #4772
https://github.com/crystal-lang/crystal/pull/4772 does.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/issues/4840#issuecomment-323134853,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAG3hKsjxvVF_UFDHj6as-V983vuodiiks5sZHMYgaJpZM4O6KAw
.