Chapel: Confusing error on object construction from JSON

[buddha314]

My goal is to populate at date-time field in a class from a JSON call. On the way to that, I get this error. Please, if you can address my ultimate goal in a different way, just close this ticket.

use DateTime;

class Fighter {
  var subclass:string;
  var level:int;
  var bday: datetime;
}

var d = datetime.now():string;
var mem = openmem();
var writer = mem.writer().write('{"subclass":"ninja", "level":7, "bday":d}');
var reader = mem.reader();

var f = new Fighter();
reader.readf("%jt", f);

writeln(f);

Results in

 chpl jsp.chpl 
$CHPL_HOME/modules/standard/IO.chpl:3090: error: unresolved call 'atomic_int64.readThis(channel(false,dynamic,false))'
jsp.chpl:3: note: candidates are: Fighter.readThis(f)
$CHPL_HOME/modules/internal/CString.chpl:235: note:                 c_string.readThis(x)
$CHPL_HOME/modules/internal/String.chpl:532: note:                 string.readThis(f)
$CHPL_HOME/modules/internal/ChapelRange.chpl:2111: note:                 ref range.readThis(f)
$CHPL_HOME/modules/internal/ChapelSyncvar.chpl:143: note:                 _syncvar.readThis(x)
$CHPL_HOME/modules/internal/ChapelSyncvar.chpl:658: note:                 _singlevar.readThis(x)
$CHPL_HOME/modules/internal/ChapelSyncvar.chpl:979: note:                 aligned_t.readThis(f)
$CHPL_HOME/modules/internal/ChapelArray.chpl:1708: note:                 _domain.readThis(f)
$CHPL_HOME/modules/internal/ChapelArray.chpl:2544: note:                 _array.readThis(f)

Note, the example comes from this SO

[marcoscleison]

Sadly, this issue is really making things difficult to serialize json request with date fields.
Before pull from master today I could serialize well the datetime type into json without issue. But now I have the same problem.

[bradcray]

I'm hoping @daviditen and @ronawho can take a quick look at this when they have time (and @mppf when he gets back). The issue is that the datetime field uses a Shared field which in turn contains an atomic field for reference counting. And atomic types don't currently support readThis() methods as the following test shows:

var x: atomic int;
read(x);

Offhand, though, I'm not sure whether the best solution is to:

• add a readThis() method to atomic types
• add a readThis() method in the Shared type to skip over the atomic field
• fix something about forwarding to make reading a Shared(t) more like reading a t, bypassing the atomic field that way
• add a readThis() method to the datetime type itself to bypass the Shared field

[bradcray]

Before pull from master today I could serialize well the datetime type into json without issue. But now I have the same problem.

@marcoscleison : When was your previous most recent pull from master? I would guess that the PR that changed this behavior was #8135 merged on January 10th so am curious if this is the first time you've updated to master since then. Thanks.

[daviditen]

I think a readWriteThis() for datetime would be useful. Also for date and time.

Adding this function to the code above and then changing the var writer = ... line slightly gets it working:

proc datetime.readWriteThis(f) {
  var dash  = new ioLiteral("-"),
      colon = new ioLiteral(":");
  f <~> new ioLiteral("{") <~> chpl_date.chpl_year <~> dash
    <~> chpl_date.chpl_month <~> dash <~> chpl_date.chpl_day
    <~> new ioLiteral(" ") <~> chpl_time.chpl_hour <~> colon
    <~> chpl_time.chpl_minute <~> colon <~> chpl_time.chpl_second
    <~> new ioLiteral(".") <~> chpl_time.chpl_microsecond
    <~> new ioLiteral("}");
}

var writer = mem.writer().write('{"subclass":"ninja", "level":7, "bday":' + d + '}');

[buddha314]

Wow, that is a deep track I would never have found. What does <~> do, anyway? Can we push this into the language rather doing it every time?

[daviditen]

<~> is an I/O operator. It either reads from or writes to a channel depending on context.

[bradcray]

Can we push this into the language rather doing it every time?

From his comment above, I assume @daviditen is thinking we'd add readWriteThis() routines to date, time, and datetime but just hasn't gotten to it yet (and/or is waiting for further approval from others). True? (It sounds good to me in any case, and thanks for looking into this, David).

[mppf]

I'm for @daviditen's approach of adding readWriteThis to date/time/datetime.

[marcoscleison]

@bradcray , you are right. I was without update my master branch for a long time. But now I am with chpl version 1.17.0 pre-release (33847f5d0b).

I tested just now and I had the same problem.

My error continues to be:

$CHPL_HOME/modules/standard/IO.chpl:3064: error: unresolved call 'atomic_int64.readThis(channel(false,dynamic,false))'

[bradcray]

Hi @marcoscleison : Yes, I didn't mean to imply that it was fixed on master now, simply that I think this behavior has been broken on master for some time now. It sounds like this is ours to fix by introducing the readWriteThis() functions that David proposes above (and that one can work around it for the time being by including his readWriteThis function within your source...)

[marcoscleison]

Hi @bradcray ,
yeah, I understand now.
I will try it.
Thank you!

[daviditen]

https://github.com/chapel-lang/chapel/pull/8787 (not merged yet at the moment) adds readWriteThis methods for date, time, and datetime.

[buddha314]

Look at how fast that man moves! I'm imagining his pony tail whipping wildly back and forth!

[daviditen]

8787 is merged. I modified the var writer = line in original example slightly:

use DateTime;

class Fighter {
  var subclass:string;
  var level:int;
  var bday: datetime;
}

var d = datetime.now():string;
var mem = openmem();
var writer = mem.writer().writef('{"subclass":"ninja", "level":7, "bday":%jt}', d);
var reader = mem.reader();

var f = new Fighter();
reader.readf("%jt", f);

writeln(f);

And it prints something like:

{subclass = ninja, level = 7, bday = 2018-03-14T15:38:50.818337}

[buddha314]

Oh cool!

[buddha314]

Will it fill in the json in order, then? if I have bday and dday, then I'd give it two arguments?

[daviditen]

Right, you could add var dday: datetime field to Fighter, then write it as:

var writer = mem.writer().writef('{"subclass":"ninja", "level":7, "bday":%jt, "dday":%jt}', d, d2);

Or if you had a Fighter instance with the fields already set you could just do:

mem.writer().writef("%jt", myFighter);