K6: Output response times in seconds

[PerfTestLIT]

The response times are output in millliseconds, would it be possible to configure the response times to be output in seconds please?

Thanks

[liclac]

Times should be output as seconds if they're higher than 1s.

[V3ckt0r]

Hey @liclac,

I was going to log something similar. At the moment the way the results come back as different units makes it none uniform and difficult when capturing and graphing the data. There are also further confusion cause via the documentation here that states these metrics should be in ms. Please see example load test below:

k6 run script.js -v

          /\      |‾‾|  /‾‾/  /‾/
     /\  /  \     |  |_/  /  / /
    /  \/    \    |      |  /  ‾‾\
   /          \   |  |‾\  \ | (_) |
  / __________ \  |__|  \__\ \___/ .io

DEBU[0000] Loading...                                    name=/home/abdib01/go/src/github.com/loadimpact/k6/script.js pwd=/home/abdib01/go/src/github.com/loadimpact/k6
DEBU[0000] Resolved...                                   name=/home/abdib01/go/src/github.com/loadimpact/k6/script.js
DEBU[0000] Babel: Transformed                            t=138.671785ms
DEBU[0001] Local: Setting max VUs                        max=10
DEBU[0001] Local: Setting VUs                            vus=10
DEBU[0001] Local: Setting paused                         paused=false
DEBU[0001] Local: Setting end time                       d=30s
DEBU[0001] Local: Setting end iterations                 i=-1
  execution: local
     output: -
     script: script.js

    duration: 30s, iterations: -
         vus: 10,  max: 10

DEBU[0001] Engine: Starting with parameters...
DEBU[0001]  - end conditions (if any)                    time=30s
DEBU[0001] Local: Scaling...                             num=10
DEBU[0001] Couldn't send usage blip                      error="Post http://k6reports.loadimpact.com/: malformed HTTP status code \"text/html\""

    ✓ protocol is HTTP/2
    ✓ status is 200

    checks................: 100.00% ✓ 33716 ✗ 0
    data_received.........: 2.8 MB  93 kB/s
    data_sent.............: 12 kB   394 B/s
    http_req_blocked......: avg=41.32µs min=0s       med=0s     max=74.77ms  p(90)=0s      p(95)=0s
    http_req_connecting...: avg=4.19µs  min=0s       med=0s     max=8.43ms   p(90)=0s      p(95)=0s
    http_req_duration.....: avg=17.6ms  min=6.83ms   med=6.86ms max=254.82ms p(90)=26.38ms p(95)=27.55ms
    http_req_receiving....: avg=7.29ms  min=465.19µs med=1.35ms max=245.76ms p(90)=16.33ms p(95)=16.79ms
    http_req_sending......: avg=58.49µs min=8.26µs   med=8.84µs max=60.99ms  p(90)=36.1µs  p(95)=42.87µs
    http_req_waiting......: avg=10.24ms min=2.31ms   med=4.79ms max=178.55ms p(90)=11.92ms p(95)=15.66ms
    http_reqs.............: 16858   561.913497/s
    iterations............: 16858   561.913497/s
    vus...................: 10      min=10  max=10
    vus_max...............: 10      min=10  max=10

The results come back as a mix between ms and µs. Could we not standardise these results and put everything into one unit scale µs? This will make it far easier to work with when gathering and graphing these metrics values.

[ghost]

Hi @V3ckt0r. Will TSDB not convert the unit of measurements itself? I think this happens with InfluxDB. The problem I think would be there may be some averaging in the timeseries database which could skew the results.

[V3ckt0r]

Yes @aidylewis you can convert it yourself. However you will have to keep changing the transformation for that to happen. When you run tests under different conditions, shorter/less intensive tests are more likely to have µs units, where as longer/more intensive tests will more likely pass the 1 second threshold thus be in ms. It becomes problematic and less streamlined if you need to do different transformations to the same metric under different test conditions.

[ghost]

This is going back some time but I think I remember InfluxDB automatically moving from milliseconds to seconds (for example) when visualising through Grafana.

[V3ckt0r]

Here is a further example to demonstrate my last comment. I've run the same test as my initial post, but increased the VUS by 20 and duration to 1min:

    data_received.........: 25 kB  413 B/s
    data_sent.............: 3.0 kB 49 B/s
    http_req_blocked......: avg=730.28µs min=1.72µs  med=1.43µs  max=43.49ms p(90)=5.27µs   p(95)=8.26µs
    http_req_connecting...: avg=665.53µs min=0s      med=0s      max=39.1ms  p(90)=0s       p(95)=0s
    http_req_duration.....: avg=39.01ms  min=38.13ms med=19.34ms max=62.79ms p(90)=39.54ms  p(95)=40.33ms
    http_req_receiving....: avg=205.94µs min=60.59µs med=59.23µs max=18.65ms p(90)=219.17µs p(95)=295.93µs
    http_req_sending......: avg=24.22µs  min=9.92µs  med=8.36µs  max=1.37ms  p(90)=37.48µs  p(95)=49.75µs
    http_req_waiting......: avg=38.78ms  min=38.02ms med=19.26ms max=51.8ms  p(90)=39.33ms  p(95)=39.99ms
    http_reqs.............: 1740   28.999921/s
    iterations............: 1710   28.499922/s
    vus...................: 30     min=30 max=30
    vus_max...............: 30     min=30 max=30

Original results from above post:

data_received.........: 2.8 MB  93 kB/s
    data_sent.............: 12 kB   394 B/s
    http_req_blocked......: avg=41.32µs min=0s       med=0s     max=74.77ms  p(90)=0s      p(95)=0s
    http_req_connecting...: avg=4.19µs  min=0s       med=0s     max=8.43ms   p(90)=0s      p(95)=0s
    http_req_duration.....: avg=17.6ms  min=6.83ms   med=6.86ms max=254.82ms p(90)=26.38ms p(95)=27.55ms
    http_req_receiving....: avg=7.29ms  min=465.19µs med=1.35ms max=245.76ms p(90)=16.33ms p(95)=16.79ms
    http_req_sending......: avg=58.49µs min=8.26µs   med=8.84µs max=60.99ms  p(90)=36.1µs  p(95)=42.87µs
    http_req_waiting......: avg=10.24ms min=2.31ms   med=4.79ms max=178.55ms p(90)=11.92ms p(95)=15.66ms
    http_reqs.............: 16858   561.913497/s
    iterations............: 16858   561.913497/s
    vus...................: 10      min=10  max=10
    vus_max...............: 10      min=10  max=10

If you take the http_req_receiving metric and look at p(95) you can see one is in µs and the other is in ms

[ppcano]

There are also further confusion cause via the documentation here that states these metrics should be in ms.

@V3ckt0r Thanks for pointing this out. We will clarify it on the doc.

ms is the unit for the response.timings.{blocked, looking_up, connecting...} properties in the JS code.

var res = http.get("http://httpbin.org");
console.log("Response time was " + String(res.timings.duration) + " ms");

But the current behavior of the console output is to show the unit depending on its value.

The results come back as a mix between ms and µs. Could we not standardise these results and put everything into one unit scale µs?

For the console output, I find better the current behavior, but it's only one opinion.

This will make it far easier to work with when gathering and graphing these metrics values.

When visualizing your results, instead of the console output, you will likely use one of the 3 collectors described in the Results output doc (code available at the stats folder). The collector should always report the metric on the same unit or in a way that the metric can be processed properly.

[liclac]

I can definitely see a case for fixing the unit when outputting time metrics and writing out fractions instead. Which unit though? 0.21s? 21.4ms?

(My personal fave is ms, I don't think many will be dealing with durations higher than 1s.)

[PerfTestLIT]

I agree that ms would be best. I opened this ticket asking timings to be output in seconds but having discussed further with dev’s and further demo’s we won’t be dealing much with durations over 1 sec and the preference is to have the timings in ms

Thanks

From: Emily Ekberg [mailto:[email protected]]
Sent: 29 November 2017 04:02
To: loadimpact/k6 [e-mail]
Cc: Braniff, Kevin (LIT) [e-mail]; Author [e-mail]
Subject: Re: [loadimpact/k6] Output response times in seconds (#363)

I can definitely see a case for fixing the unit when outputting time metrics and writing out fractions instead. Which unit though? 0.21s? 21.4ms?

(My personal fave is ms, I don't think many will be dealing with durations higher than 1s.)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/loadimpact/k6/issues/363#issuecomment-347746303, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AgILCs9rf0CSpdSDWeg6Be802LBkCOHEks5s7NdNgaJpZM4QexKp.

[V3ckt0r]

Yup, ms would also make it more consistent with other tools in the open source ecosystem.