Csvhelper: Regression reading CSV (phantom extra field) after upgrade from v19 to v21

[nil4]

Describe the bug
After upgrading a project from CsvHelper 19.0.0 to 21.0.6, a functional regression was observed, where reading valid CSV returns an extra field that is not present in the source, for one of the CSV rows, but not others.

The issue reproduces with 21.1.0 as well.

Due to breaking changes in 21.x, it may be that the APIs used are incorrect, hence providing a self-contained runnable repro with code and input CSV data at https://github.com/nil4/csvhelper-regression-19-to-21.

To Reproduce
Steps to reproduce the behavior:

1. Clone https://github.com/nil4/csvhelper-regression-19-to-21
2. Run repro.cmd, which builds and runs the project three times (with CsvHelper 19.0.0, 21.0.6 and 21.1.0, respectively), each time reading the same CSV input file.

Expected behavior
Every CsvHelper version successfully reads all 5136 records in the input file, and each record has exactly 6 fields.

Observed behavior
CsvHelper 19.0.0 reads the complete input file correctly, while 21.x versions return an extra 7^th field for record # 5014:

    field #6: f050c1cf477dc603
    field #7: f050c1cf477dc603\r

Screenshots
The output produced by repro.cmd is similar to (modulo MSBuild summaries and newlines, removed for brevity):

===============================
Building with CsvHelper v19.0.0
Build succeeded.
-------------------------------
Testing with CsvHelper v19.0.0
Successfully read 5136 records from ReproData.csv
---- CsvHelper v19.0.0 -- OK

===============================
Building with CsvHelper v21.0.6
Build succeeded.
-------------------------------
Testing with CsvHelper v21.0.6

Shouldly.ShouldAssertException: record.Length
    should be 6
    but was 7
Additional Info: record[5014].Length != fieldNames.Length

Reading record #5014 of ReproData.csv, found:
    field #1: 5862
    field #2: 
    field #3: 2020-11-23T15:38:33.00Z
    field #4: 
    field #5: 0
    field #6: f050c1cf477dc603
    field #7: f050c1cf477dc603\r

!!!! CsvHelper v21.0.6 -- FAILED !!

===============================
Building with CsvHelper v21.1.0
Build succeeded.
-------------------------------
Testing with CsvHelper v21.1.0

Shouldly.ShouldAssertException: record.Length
    should be 6
    but was 7
Additional Info: record[5014].Length != fieldNames.Length

Reading record #5014 of ReproData.csv, found:
    field #1: 5862
    field #2: 
    field #3: 2020-11-23T15:38:33.00Z
    field #4: 
    field #5: 0
    field #6: f050c1cf477dc603
    field #7: f050c1cf477dc603\r

!!!! CsvHelper v21.1.0 -- FAILED !!

Additional context

OS: Windows 10.0.19042 (x64)

dotnet --info

.NET SDK (reflecting any global.json):
 Version:   5.0.102
 Commit:    71365b4d42

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19042
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.102\

[JoshClose]

Fix in 21.1.1 on NuGet. https://www.nuget.org/packages/CsvHelper/21.1.1

[nil4]

@JoshClose I really appreciate the quick turn-around time!

Unfortunately, testing reveals that an issue (albeit a different one) is still present.

After adding 21.1.1 to the test scenario in https://github.com/nil4/csvhelper-regression-19-to-21/commit/a0a7dc8b82cac73c40f915c138612d6f9ec77304, running repro.cmd still fails with:

===============================
Building with CsvHelper v21.1.1
Build succeeded.
-------------------------------
Testing with CsvHelper v21.1.1

Unexpected exception reading record #5014
System.ArgumentOutOfRangeException: Length cannot be less than zero. (Parameter 'length')
   at System.String.Ctor(Char[] value, Int32 startIndex, Int32 length)
   at CsvHelper.CsvParser.get_Record()
   at Program.ReadOneRecord(CsvReader csv) in Repro.cs:line 87
   at Program.Main() in Repro.cs:line 31


!!!! CsvHelper v21.1.1 -- FAILED !!

Since this occurs at the same record (5014) as before I think that this is related. Do you think I should I open a new issue about this?

[JoshClose]

I know exactly what this is. I created a unit with a small set of data and didn't re-run your code. I will do that this time. :)

[JoshClose]

Fixed in 21.1.2 on NuGet. https://www.nuget.org/packages/CsvHelper/21.1.2

Your example parses fine this time. ;) This could have been an issue when the buffer ran out in a delimiter or quote also, so this was a good find.

[nil4]

@JoshClose I am happy to report that the issue is confirmed resolved with 21.2.0 -- both the self-contained repro, and the original app where this was found behave as expected! 👍

Thank you for the swift resolution time -- much appreciated!

[JoshClose]

Excellent! Thanks for the detailed info. It really helped.