Chapel: What should happen when a promoted expression is returned?

[mppf]

When returning a promoted expression, a reasonable question to ask is when the promoted expression should be evaluated. Consider for example the following program, which promotes add. Should "In add" be printed after "In f" or after "In g"? (In other words, should the promoted expression be evaluated in the process of returning from f() ? Or should it be evaluated in the process of being assigned to C?).

var A:[1..4] int;
for i in 1..4 do
  A[i] = i;
var B:[1..4] int;
for i in 1..4 do
  B[i] = 10*i;

proc add(a: int, b: int) {
  writeln("In add");
  return a + b;
}

proc f() {
  writeln("In f");
  return add(A, B);
}

proc g(x) {
  writeln("In g");
  A[2] = 1;

  var C = x;
  writeln(C[2]); // Does this print 21 or 22?
}

g(f());

On master today, f returns an iterator record that contains the promoted expression. That causes the evaluation of add to be put off until the initialization of C in g.

This strategy has the downside that it is unclear how to correctly compile a program such as the following, in which the promoted iterator captures a local variable:

var A:[1..4] int;
for i in 1..4 do
  A[i] = i;

proc add(a: int, b: int) {
  writeln("In add");
  return a + b;
}

proc f() {
  writeln("In f");
  var B:[1..4] int;
  for i in 1..4 do
    B[i] = 10*i;
  return add(A, B);
}

proc g(x) {
  writeln("In g");
  A[2] = 1;

  var C = x;
  writeln(C[2]); // does it print out 22 or 21?
}

g(f());

This program prints neither 21 or 22 on master today (and has memory access errors instead).

Associated Future Test(s):

test/arrays/ferguson/semantic-examples/1-return-promoted-expr.chpl
test/arrays/ferguson/semantic-examples/valgrind-futures/4-return-promoted-expr-local.chpl
test/arrays/ferguson/semantic-examples/valgrind-futures/4-return-promoted-expr-records.chpl

[mppf]

Given that arrays return by value, it makes sense to me thatreturn add(A,B) should return an array value. In other words, var x = add(A,B); return x; should not change the behavior.

[mppf]

@benharsh - this is the issue we were discussing.

[bradcray]

My intuition is that the promoted expression should be evaluated prior to returning.

[mppf]

While I'm sure I could adjust the compiler differently, my current implementation that evaluates the promoted expression (and stores it in an array) prior to returning also catches returning a call to an iterator (what the compiler currently considers to amount to returning an iterator record). For example:

iter myIter() {
  writeln("starting myIter");
  for i in 1..3 {
    writeln("yielding ", i);
    yield i;
  }
  writeln("done myIter");
}

proc returnsMyIter() {
  return myIter();
}

for x in returnsMyIter() {
  writeln(x);
}

on master it prints

starting myIter
yielding 1
1
yielding 2
2
yielding 3
3
done myIter

but on my branch it prints

starting myIter
yielding 1
yielding 2
yielding 3
done myIter
1
2
3

On my branch, there is a pragma available for our internal use that disables converting the iterator to an array on return, so it's possible to get the old behavior with the pragma. However I think from a language design point of view, returning an iterator should be the same as returning a promoted expression. (Returning myIter() seems to be a kind of degenerate case of returning a promoted expression identity(myIter()) for example).

Thoughts?

[mppf]

P.S. The error https://github.com/chapel-lang/chapel/blob/master/compiler/passes/checkResolved.cpp#L65 "functions cannot return nested iterators or loop expressions" is surely related to this decision. I see this error as a nod to the fact that there are real memory management challenges to allowing returning an iterator that refers to local variables in a function (that will be out of scope / destroyed / stack trash by the time the iterator runs).

[bradcray]

Thoughts?

I lost track of this issue over jury duty and the holidays. I agree that this is consistent with our reactions to the other case and that the new behavior is at least as good as the old.