Chapel: Should single locale libraries do something special with endCounts?

[lydia-duncan]

Executables call a generated function, chpl_gen_main, which performs some runtime calls (e.g. chpl_rt_preUserCodeHook) and other set up and tear down operations that depend on types that are generated by the compiler (including endCounts). Libraries instead call chpl_library_init, the functions requested by the library's client, and chpl_library_finalize - these perform some of the runtime calls but not all of them.

Because chpl_library_init and chpl_library_finalize are defined in the runtime, they cannot include references to endCounts - this may mean that we fail to appropriately clean up after begin statements in exported functions, for instance. This seems like something that chpl_library_finalize should accomplish (and thus something that chpl_library_init may need to set up for it to do so), but I'm not sure how.

One idea is to generate this particular set of operations into the module initialization and deinitialization function of a user module (though the library may contain multiple modules so there is a question about which one should be modified), or of an internal module like ChapelStandard. Note that doing so would not be appropriate for multilocale libraries as they stand today (see #13240) because the server side performs these actions through chpl_gen_main - that said, we could update multilocale libraries to be in step with single locale libraries if we decide to take this change on.

I don't have other ideas at this point in time.

[lydia-duncan]

Note: I don't know the priority of this right now. My thinking is to wait to think about it more until someone actually runs into problems with it, but I don't know if that is wise.

[lydia-duncan]

Audrey may have just run into this

[astatide]

I should construct an example that shows this, but basically: compiling a python library that makes use of the include statement will bork out when you call said begin.

[dlongnecke-cray]

An example would be great, thanks!

[astatide]

"When's the best time to write code to showcase this problem, Audrey?"
"21 days ago was the best. Second best time is now."

https://gist.github.com/astatide/f843c0c243e0f146ae0aa52b637d12ce

[ben-albrecht]

We effectively need to set up the endcount and the dynamic endcount routine in chpl_setup (after runtime init), and then call _waitEndCount from chpl_cleanup (before runtime deinit).

The work-around posted in #16583 provides a model of what we might want to start generating in chpl_setup and chpl_cleanup:

use Time;

// This could be generated as part of chpl_init
export proc initDynamicEndCount() {
  var _endCount = _endCountAlloc(false);
   __primitive("set dynamic end count", _endCount);
}

// This could be generated as part of chpl_cleanup
export proc waitDynamicEndCount() {
  var _endCount = __primitive("get dynamic end count");
  _waitEndCount(_endCount);
  _endCountFree(_endCount);
}

export proc hello() {
  writeln('hello');
  begin sleepTest();
  writeln('world');
}

proc sleepTest() {
  writeln('before sleep');
  sleep(1);
  writeln('after sleep');
}

from foo import foo

foo.chpl_setup()
foo.initDynamicEndCount()

print('calling foo.hello()')
foo.hello()

foo.waitDynamicEndCount()
# implicitly calling foo.chpl_cleanup()