K6: Summary report in json format is missing fields defined in summaryTrendStats

[alexandr-lobyntsev]

The summary report in stdout has fields provided by summaryTrendStats, but in JSON format report ignores that definition

Environment

K6 version: k6 v0.27.1 (2020-07-30T10:20:27+0000/v0.27.1-0-g4ee1ca96, go1.14.6, linux/amd64)
OS version: 18.04.2 LTS (Bionic Beaver)

Expected Behavior

The fields defined in summaryTrendStats are represented in the JSON summary report

Actual Behavior

The fields defined in summaryTrendStats are ignored during the JSON summary report generation

Steps to Reproduce the Problem

1. Define custom percentiles p(98) with the options

export let options = {
  summaryTrendStats: ["avg", "min", "med", "max", "p(90)", "p(95)", "p(98)"],

1. Execute k6 run with the --summary-export agrument to generate JSON summary report

Stdout summary report

    http_req_blocked.........................: min=1.02µs   avg=821.07µs med=5.84µs   max=378.81ms p(90)=9.12µs   p(95)=10.97µs  p(98)=21.87µs 
    http_req_connecting......................: min=0s       avg=152.8µs  med=0s       max=85.03ms  p(90)=0s       p(95)=0s       p(98)=0s      
    http_req_duration........................: min=45.3ms   avg=58.76ms  med=53.99ms  max=619.42ms p(90)=66.84ms  p(95)=75ms     p(98)=99.86ms 
    http_req_receiving.......................: min=16.66µs  avg=247.08µs med=100.55µs max=49.85ms  p(90)=596.83µs p(95)=723.86µs p(98)=1.1ms   
    http_req_sending.........................: min=6.18µs   avg=45.21µs  med=33.13µs  max=7.32ms   p(90)=47.98µs  p(95)=69.18µs  p(98)=271.45µs
    http_req_tls_handshaking.................: min=0s       avg=351.39µs med=0s       max=250.02ms p(90)=0s       p(95)=0s       p(98)=0s      
    http_req_waiting.........................: min=45.21ms  avg=58.47ms  med=53.74ms  max=619.32ms p(90)=66.48ms  p(95)=74.81ms  p(98)=98.68ms 
    http_reqs................................: 9930    654.441841/s
    iteration_duration.......................: min=149.28ms avg=190.69ms med=174.06ms max=750.84ms p(90)=214.95ms p(95)=262.53ms p(98)=467.06ms
    iterations...............................: 1986    130.888368/s

JSON summary report

        "http_req_blocked": {
            "avg": 0.8210748100704939,
            "max": 378.813427,
            "med": 0.005846,
            "min": 0.001024,
            "p(90)": 0.0091233,
            "p(95)": 0.010976199999999997
        },
        "http_req_connecting": {
            "avg": 0.15280677905337361,
            "max": 85.035816,
            "med": 0,
            "min": 0,
            "p(90)": 0,
            "p(95)": 0
        },
        "http_req_duration": {
            "avg": 58.76745789798619,
            "max": 619.42353,
            "med": 53.996348,
            "min": 45.30651,
            "p(90)": 66.8408204,
            "p(95)": 75.0026239
        },
        "http_req_receiving": {
            "avg": 0.24708492749244615,
            "max": 49.858361,
            "med": 0.1005595,
            "min": 0.016666,
            "p(90)": 0.5968348000000002,
            "p(95)": 0.7238643999999999
        },

[na--]

Thanks for reporting this! It was previously reported in https://github.com/loadimpact/k6/issues/1433, but I'll keep this issue open and close the other one, since this has more details.

[na--]

Also, this is potentially a connected issue, though maybe not quite the same: https://github.com/loadimpact/k6/issues/1253

[alavi201]

Hi @na-- do you have any updates on this issue?

[na--]

Yes, it's likely we'll fix this for k6 v0.30.0, since it's connected to other end-of-test summary issues we're currently working on.

[alexandr-lobyntsev]

Yes, it's likely we'll fix this for k6 v0.30.0, since it's connected to other end-of-test summary issues we're currently working on.

We are waiting for it