Specflow: Error: Could not generate test run result reports. An error occurred in the generator.

[xDazedx]

SpecFlow Version:

• [x] 3.0

Used Test Runner

• [x] SpecFlow+Runner

Version number: 3.1.33

Project Format of the SpecFlow project

• [x] Classic project format using packages.config

.feature.cs files are generated using

• [x] SpecFlow.Tools.MsBuild.Generation NuGet package

Visual Studio Version

• [ ] VS 2019
• [x] VS 2017

Enable SpecFlowSingleFileGenerator Custom Tool option in Visual Studio extension settings

• [ ] Enabled
• [x] Disabled

Are the latest Visual Studio updates installed?

• [x] Yes
• [ ] No, I use Visual Studio version <Major>.<Minor>.<Patch>

.NET Framework:

• [x] >= .NET 4.5
• [ ] before .NET 4.5
• [ ] .NET Core 2.0
• [ ] .NET Core 2.1
• [ ] .NET Core 2.2
• [ ] .NET Core 3.0

Test Execution Method:

• [x] Visual Studio Test Explorer
• [ ] TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
• [ ] Command line – PLEASE SPECIFY THE FULL COMMAND LINE

<SpecFlow> Section in app.config or content of specflow.json

Issue Description

When executing a test case locally, we are not seeing the report get generated.
output:
[11/19/2019 3:17:47 PM Informational] test run finished
[11/19/2019 3:17:47 PM Informational] publishing test results
[11/19/2019 3:17:47 PM Informational] test results published
[11/19/2019 3:17:47 PM Informational] generating reports
[11/19/2019 3:17:47 PM Informational] Starting external report generation process
[11/19/2019 3:17:48 PM Informational] Could not generate test run result reports. An error occurred in the generator.
[11/19/2019 3:17:48 PM Informational] External program returned exit code '-1'
[11/19/2019 3:17:48 PM Informational] Failed generating reports
[11/19/2019 3:17:48 PM Informational] Result: all tests passed
[11/19/2019 3:17:48 PM Informational] Total: 1

Steps to Reproduce

[SabotageAndi]

When you upgraded to SpecFlow+Runner 3, did you adjusted the report template?
See https://specflow.org/2019/updating-to-specflow-3/ - SpecFlow+ Runner Report Templates

[xDazedx]

Hey Andi - We are using the default report and not calling a custom report. The odd thing is that this is only local runs. Reports out of Jenkins are fine.

[SabotageAndi]

@xDazedx Could you send us your specrun.log where the error happens?

[xDazedx]

Andi here are the two logs I see

---

2019-11-20T10:24:41.7538022-07:00;Thread#13;Info;generating reports
2019-11-20T10:24:41.8705218-07:00;Thread#13;Info;Starting external report generation process
2019-11-20T10:24:42.6464132-07:00;Thread#13;Error;Could not generate test run result reports. An error occurred in the generator.
2019-11-20T10:24:42.6464132-07:00;Thread#13;Error;External program returned exit code '-1'
2019-11-20T10:24:42.6464132-07:00;Thread#13;Info;Failed generating reports

---

2019-11-20T10:24:42.2704170-07:00;Thread#1;Info;Reading test run result from Json file "C:UsersAppDataLocalTemptmpD988.tmp".
2019-11-20T10:24:42.2744085-07:00;Thread#1;Info;Successfully read test run result from Json file "C:UsersAppDataLocalTemptmpD988.tmp"
2019-11-20T10:24:42.2744085-07:00;Thread#1;Info;Deserializing test run result
2019-11-20T10:24:42.6174928-07:00;Thread#1;Error;System.Exception thrown during deserialization: System.Exception: Not a FilterItem.
at TechTalk.SpecRun.Framework.JsonConverterExtensions.DeserializeToFilterItem(JObject jsonObject)
at TechTalk.SpecRun.Framework.JsonConverterExtensions.DeserializeT
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonSerializer.DeserializeT
at TechTalk.SpecRun.Framework.Services.JsonNetSerializer.DeserializeT
2019-11-20T10:24:42.6184900-07:00;Thread#1;Error;Test run result could not be loaded. The content of "C:UsersAppDataLocalTemptmpD988.tmp" is not a valid JSON test run result.
2019-11-20T10:24:42.6184900-07:00;Thread#1;Error;Could not fetch the test run results.
========== End of log file ==========

[SabotageAndi]

Thanks for the log entries. I was able to find the issue.
It's fixed and build/release pipelines are running.

[xDazedx]

Appreciate it Andi.

[SabotageAndi]

NuGet package with fix are uploaded to NuGet.org.

[liamharries]

@SabotageAndi not sure if I'm experiencing the same issue or not

2019-11-26T15:12:10.2713398+00:00;Thread#12;Error;Could not generate test run result reports. An error occurred in the generator.
2019-11-26T15:12:10.2717206+00:00;Thread#12;Error;External program returned exit code '-1'
2019-11-26T15:12:10.2718728+00:00;Thread#12;Info;Failed generating reports

I've just created a new project using the latest packages but can't for the life of me work out how to generate the report.

The only difference in my setup from above is I'm using .NET Core 2.1 and VS2019.

[SabotageAndi]

@liamharries please create a new issue here on GitHub.

[liamharries]

@SabotageAndi my issue was fixed by the linked issue 1793 and version 3.1.37

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.