K6: Option 'throw' is broken or badly documented

[marklagendijk]

Expected behaviour

Enabling throw exits the specific test iteration as soon as an request error occurs: connection issues, status 4xx, status 5xxx.

I hoped that with this option I wouldn't have to write checks for every request.

Actual behaviour

Status errors don't exit the iteration, or show anything in the log.

Documented behaviour

A boolean, true or false, specifying whether to throw errors on failed HTTP requests or not.

[cuonglm]

It depends on how you defined failed request.

You can load test an endpoint which return 404 or 400. If you expect it to return 400, but it return 200, then it's a failed request in scope of your test.

[marklagendijk]

In the end I assumed that that is indeed what the documentation meant.
However, that is not what I, as a test developer, would assume it means when reading the documentation.

Apart from the documentation I also think that the feature would make much more sense when it also applied to 4xx and 5xx errors:

• In most tests every request should return a success status 2xx.
• In load tests it can happen that one overloads the system. In such case you usually get a 503 error.
• If the feature would also cover error statuses, you could just enable it, without having to worry about writing a check for _every_ request.

[cuonglm]

@markjmeier I looked at the source code again, and the term of failed request is not about 4xx and 5xx status code, but k6 can't connect to remote host.

```
if resErr != nil {
// Do not log errors about the contex being cancelled.
select {
case <-ctx.Done():
default:
state.Logger.WithField("error", resErr).Warn("Request Failed")
}

    if preq.Throw {
        return nil, resErr
    }
}```

[na--]

Thanks for mentioning this, we'll improve the docs for throw.

But the use case of throwing errors when the connection can't be established (or there's a DNS error, TLS error, etc. non-application errors) or when the remote server returns a response with a 4xx/5xx (application error) status is a _very_ valid use case. Unfortunately, the current global throw option is too inflexible, so I'm thinking that this is also something we should handle in the new HTTP API, so I'll link it that new issue.