K6: DNS round-robin ignored, always uses the same host

[stephank]

k6 calls FetchOne on the DNS resolver:

https://github.com/loadimpact/k6/blob/d4152d20221c0c466c93fe5cabb8e8f8c885cb5b/lib/netext/dialer.go#L67

But this only ever returns the first entry from the cache:

https://github.com/viki-org/dnscache/blob/c70c1f23c5d84ab39054ec6b418a5ed0fa51cade/dnscache.go#L37

Our HAProxy frontends individually limit requests, and our load test was exceeding those limits, while the entire HAProxy cluster should've been able to handle the load. The error message k6 logged showed it was using the same host for all requests. (Or at least, the failing requests.)

As test and temporary workaround, I've adjusted our script to list the IPs and pick one at random for each VU, and the load test is now fine. (For this, we had to use --insecure-skip-tls-verify. This'd also break vhosts, but that wasn't a problem for us. And maybe that can also be worked around by setting Host manually.)

[na--]

This is very loosely connected to https://github.com/loadimpact/k6/issues/726 and it may be worth it to fix both at the same time

[ofauchon]

Hi,

How would you like these issues to be solved (#726 / #738) ?
I can see multiple solutions:

• Propose patches for inclusion in github.com/viki-org/dnscache
• Fork github.com/viki-org/dnscache in loadimpact repositories
• Write a new, k6-specific DNS resolver for the project

Thanks;
Olivier.

[na--]

I'm not sure what the best solution would be, sorry. I'd start by investigating whether we need dnscache at all, since it would be better if the OS or the go standard library handled DNS caching in a manner that complied with TTL and could handle multiple IPs per domain.

Regarding github.com/viki-org/dnscache - it seems like it's a very small library that hasn't been updated in over 5 years, so if we have to use some form of dns caching, we should probably replace it with a more fully-featured library (if such a thing exists) or write one ourselves.

[na--]

This, together with https://github.com/loadimpact/k6/issues/726 will be fixed once we switch to the new caching DNS resolver that @bookmoons is currently woking on (https://github.com/bookmoons/resolvent/)

[brandonh-msft]

any update on this?

[na--]

@brandonh-msft, yes, actually! It was fixed in https://github.com/loadimpact/k6/pull/1612, which was almost released as a part of k6 v0.28.0. Unfortunately we didn't merge it because of some last-minute IPv6 issues we found, so it's going to be released as a part of k6 v0.29.0, slated for release mid-November.

It's likely there would also be further DNS improvements in k6 as well (https://github.com/loadimpact/k6/pull/1637), though (somewhat ironically judging by your user name), from what I understand, there are some issues with that PR on Windows, so it might be delayed until v0.30.0 or maybe only work on Linux and Mac. In any case, the round-robin fixes will be available soon in v0.29.0, and even sooner in master.

[brandonh-msft]

@brandonh-msft, yes, actually! It was fixed in #1612, which was almost released as a part of k6 v0.28.0. Unfortunately we didn't merge it because of some last-minute IPv6 issues we found, so it's going to be released as a part of k6 v0.29.0, slated for release mid-November.

k - we don't want to wait until then so we'll look at building/pulling the branch in which it was fixed

It's likely there would also be further DNS improvements in k6 as well (#1637), though (somewhat ironically judging by your user name), from what I understand, there are some issues with that PR on Windows, so it might be delayed until v0.30.0 or maybe only work on Linux and Mac. In any case, the round-robin fixes will be available soon in v0.29.0, and even sooner in master.

We're not using the client on Windows so 0.29.0 will be fine

cc @priyaananthasankar

[na--]

Closed by https://github.com/loadimpact/k6/pull/1612