Ecma262: implementation-definedness of invoking built-in functions

[jmdyck]

In the definition of [[Call]] for builtin function objects, step 10 refers to the result of evaluating _F_ in an implementation-defined manner that conforms to the specification of _F_. The use of "implementation-defined" means that:

1. the resulting behavior may vary between implementations, and
2. the implementation must document the "manner" in which it evaluates _F_.

But:

1. Do we want variation in the behavior of invoking built-in functions? Wouldn't that be an interoperability disaster?
2. Do implementations actually document anything on this point?

What if just deleted the phrase "implementation-defined"? So: the result of evaluating _F_ in a manner that conforms to the specification of _F_.

[zenparsing]

I take "implementation-defined manner" to mean that the algorithms can be written in a different way or in C++. I don't take it to mean that an implementation can modify the observable behavior of the algorithm.

[erights]

The text "the resulting behavior may vary between implementations" certainly sounds like it is talking about observable differences. What else does "behavior" mean?

A spec is only about observable differences. If this text is indeed only intending to clarify a non-observable something, it should either be removed, or moved to a non-normative note and rephrased to make that clear.

[jmdyck]

The text "the resulting behavior may vary between implementations" certainly sounds like it is talking about observable differences.

Note that that text is mine -- the spec doesn't actually define "implementation-defined" (or "-dependent") -- but I think it's the generally-accepted definition.

[littledan]

My interpretation matches @zenparsing 's, but this differs from other uses of the term "implementation-defined" in this specification (and in general). I wouldn't mind removing the words "implementation-defined" here, to avoid confusion.

The entire specification should be read with the interpretation that only the observable result needs to be the same, and the algorithms to implement it may differ; I believe the Algorithm Conventions section is already pretty clear about this:

The algorithms are not intended to imply the use of any specific implementation technique. In practice, there may be more efficient algorithms available to implement a given feature.

[annevk]

FWIW, I've only ever seen "implementation-defined" used as a way to allow observable behavior differences. (With performance not being considered observable behavior even though it is.)

[ljharb]

Does someone want to propose a PR to close this issue?

[allenwb]

A couple of thoughts:

A spec is only about observable differences.

Ultimately, this is true. But not everything normative in the specification is directly observable. Many normative steps in the specification are about establishing a semantic context that is necessary to define the observable behavior.

Here is the de facto standard meaning of these terms among language people:

Implementation-defined: The behavior is defined by each implementation and hence may differ among implementations. Within an implementation the behavior should be intentional and consistent. Implementations _should_ document the behavior (of course, the ultimate documentation is the code of the implementation).

Implementation-dependent: The behavior may vary among implementation. It it not required to be intentional or consistent within an implementation. Implementations don't need to document what they do. They many not even think about it or explicitly deal with such cases. Random shit happening is one form of implementation-dependent behavior.

[ljharb]

In what ways would we want the behavior of invoking to differ across implementations?

[allenwb]

It all depends might directly implement all non-ECMAScript built-ins in its internal implementation language. Another implementation might implement all its built-ins in Lisp and invoke the Lisp interpreter to evaluate the body of built-ins. Another implementation might define a general purpose "foreign function interface" and use it to install built-ins...

[ljharb]

Can you explain in what way the phrase "implementation-defined" is necessary to ensure that implementations are free to do these things?

[ljharb]

More to the point, everything not forbidden or mandated is allowed, so what in the spec (in the absence of this phrase) forbids this?

[allenwb]

Nothing, as I explained in https://github.com/tc39/ecma262/pull/1523#issuecomment-488050639

But it directs the reader to be conscience of the fact that this behavior is something that most be defined by the implementation rather than by the specification they are reading.

[allenwb]

In short, t 9.3.1 is all about specifying the things that most happen for a call out into a non-ECMAScript function . The words "implementation-defined" in step 10 are saying:this is where the implementation can do whatever is necessary to actually call out".

[ljharb]

@jmdyck @littledan @annevk @erights given the above, can we come up with another way to describe this concept without also creating confusion about whether hosts are allowed to differ in observable behavior?

[erights]

My position at https://github.com/tc39/ecma262/issues/1395#issuecomment-451512445 is unchanged:

A spec is only about observable differences. If this text is indeed only intending to clarify a non-observable something, it should either be removed, or moved to a non-normative note and rephrased to make that clear.

[allenwb]

@erights
So, what would you replace say in place of step 10 of 9.3.1https://tc39.github.io/ecma262/#sec-built-in-function-objects-call-thisargument-argumentslist)?

[erights]

Let result be the Completion Record that is the result of evaluating F ~in an implementation-defined manner that conforms to the specification of F~ . thisArgument is the this value, argumentsList provides the named parameters, and the NewTarget value is undefined.

Let result be the Completion Record that is the result of evaluating F. thisArgument is the this value, argumentsList provides the named parameters, and the NewTarget value is undefined.

[jmdyck]

Normally, when the spec refers to the result of evaluating X, it means there are algorithms elsewhere in the spec that tell you how to evaluate any X. So, with the extra wording deleted from the step in question, the reader might reasonably think that convention applies here too, and ask, "Okay, so where does the spec tell me how to evaluate a function F?" E.g., they might end up at the Evaluation operation, and find that it doesn't really apply (or else seems to lead in circles).

I think it's important to signal to the reader that the spec's usual model of interpreting source text doesn't tell you how to carry out this algorithm step.

[erights]

I think it's important to signal to the reader that the spec's usual model of interpreting source text doesn't tell you how to carry out this algorithm step.

Fair enough. How should it do that?

[jmdyck]

Well, if we wanted to push on that aspect, I'd suggest taking the words you struck through and making them a note at the end of the step, something like:

Let _result_ be the Completion Record that is the result of invoking _F_; _thisArgument_ is the this value, _argumentsList_ provides the named parameters, and the _NewTarget_ value is undefined. NOTE: Because _F_ is not an ECMAScript function, this document cannot specify how this invocation occurs. However, its behavior must conform to the specification of _F_.

I also changed "evaluating" to "invoking" to get us away from the possibly-misleading phrase the result of evaluating X.

[allenwb]

Right, the [[Call]] algorithm we are talking about is the definition of "evaluating F" so we shouldn't be circular.

Of course, "invoke", or "execute", or pretty much any other word we could put there doesn't have a normative meaning. That's why "in an implementation defined manner" was originally there. It means, that the implementation must state somewhere what it will do. Whether (or not) it conforms to what it says it will do is probably observable.

"its behavior must conform to the specification of F" is a normative requirement (whose non-conformance would likely be observable) so it shouldn't be tagged as "informative".

[jmdyck]

It means, that the implementation must state somewhere what it will do.

This goes back to one of my original questions: Do implementations actually document anything with respect to this step? If it's an extension point, they presumably do, but what if it isn't? Should we be requiring them to? It seems to me that as long as the function call's behavior is conformant, the implementation shouldn't be obliged to give details about how it accomplishes that.

"its behavior must conform to the specification of F" is a normative requirement (whose non-conformance would likely be observable) so it should be tagged as "informative".

(Presumably you mean "should not", unless I misunderstand what you mean by "tagged".)

Although it looks like a normative requirement, I see it as merely a reminder of a requirement that is stated normatively elsewhere. In this case, that's in 9.3 Built-in Function Objects, first para: "In either case, the effect of calling such functions must conform to their specifications."

(I contemplated using "Of course" instead of "However":

Of course, its behavior must conform to the specification of F.

to make it clearer that this is just a reminder, but I wasn't sure I liked how it sounded. The current spec has only one occurrence of the phrase "of course".)

[allenwb]

It means, that the implementation must state somewhere what it will do.

This goes back to one of my original questions: Do implementations actually document anything with respect to this step? If it's an extension point, they presumably do, but what if it isn't? Should we be requiring them to? It seems to me that as long as the function call's behavior is conformant, the implementation shouldn't be obliged to give details about how it accomplishes that.

The conventional meaning of "implementation-defined" (in contrast to "implementation-dependent") is that implementations are required to document what they do. Of course, there is no way to force compliance. Arguably, availability of an implementation's source code weakly satisfies the requirement. I supposed in this case "implementation-dependent" would be just as good, but I generally shy away from using "implementation-dependent" as it is often interpreted as random shit might happen here so don't do this.

"its behavior must conform to the specification of F" is a normative requirement (whose non-conformance would likely be observable) so it should be tagged as "informative".

(Presumably you mean "should _not_", unless I misunderstand what you mean by "tagged".)

yes

Although it looks like a normative requirement, I see it as merely a reminder of a requirement that is stated normatively elsewhere. In this case, that's in 9.3 Built-in Function Objects, first para: "In either case, the effect of calling such functions must conform to their specifications."

Yes, it is restating a remote normative requirement adjacent to where it applies. But it is still a normative requirement no matter how many times it is restated. Tagging it as "informative" seems confusing.

I agree that "of course" is awkward and probably should be avoided. The actual statement of the requirements in 9.3 is buried in a prose paragraph, so it is hard to concisely reference. that's probably why I repeated in in this step

>

[jmdyck]

The conventional meaning of "implementation-defined" ...

I'm not asking about the meaning of "implementation-defined". I'm fine with the definition you gave 20 days ago. I'm saying that the term should not be used at this step, because:
(a) it requires implementations to document something they shouldn't have to, and
(b) it allows behavior to differ between implementations, which we do not want here.

I supposed in this case "implementation-dependent" would be just as good,

That would eliminate (a), but not (b).

Although it looks like a normative requirement, I see it as merely a reminder of a requirement that is stated normatively elsewhere.

Yes, it is restating a remote normative requirement adjacent to where it applies. But it is still a normative requirement no matter how many times it is restated. Tagging it as "informative" seems confusing.

But that's one of the main things that Notes do: restate normative requirements, to help the reader understand what they mean or where they're significant.

[allenwb]

(b) it allows behavior to differ between implementations, which we do not want here.

Is that true? Consider for example, an implementation that support a kind of Realm where the JS code that creates the realm is allowed to selectively supply webasm functions for various well known intrinsics. Do we not want to allow this?

I'm beginning to think that the existing language is just fine. It allows for a lot of "what ifs" but in general it is understood to mean: do whatever the implement needs to do to enter and exit from a built-in function that is not an ECMAScript function.

[jmdyck]

Consider for example, an implementation that support a kind of Realm where the JS code that creates the realm is allowed to selectively supply webasm functions for various well known intrinsics. Do we not want to allow this?

Probably. But implementing built-ins via webasm code is an implementation choice, not a difference in behavior.

We're going in circles here. Above, you listed various ways that built-ins might be implemented, but later agreed that nothing in the spec would prevent such variation if "implementation-defined" were deleted from the step in question.

in general it is understood to mean: do whatever the implement needs to do to enter and exit from a built-in function that is not an ECMAScript function.

That may well be the intended meaning, but the only way to understand it that way is to ignore the definition of "implementation-defined".