Csvhelper: Problem with reading special characters, getting �

[devanshudravid]

With reference to issue: https://github.com/JoshClose/CsvHelper/issues/462

@JoshClose I am facing this issue while reading a Csv.

Configuration I have:
var csvConfig = new CsvConfiguration
{
Encoding = Encoding.UTF8,
HasHeaderRecord = true,
IgnoreHeaderWhiteSpace = true,
IsHeaderCaseSensitive = false,
IgnoreBlankLines = true,
CultureInfo = new CultureInfo("es")
};

And the Output is

Cargando anomal�as...
No se encontraron anomal�as
Descripci�n
Abrir panel del cl�ster

For
Cargando anomalías...
No se encontraron anomalías
Descripción
Abrir panel del clúster

It works when I use Encoding.GetEncoding("ISO-8859-1"), but it should work with UTF-8 too.

[AltruCoder]

UTF-8 and ISO-8859-1 are only encoded the same for the first 128 characters. After that UTF-8 uses 2 bytes and ISO-8859-1 uses 1 byte.
https://stackoverflow.com/a/7048780/2355006

[devanshudravid]

@AltruCoder I tried the same using a .txt file with same string. It worked for Stream Reader with UTF-8.

[devanshudravid]

Also my file is .csv. I can see in MS Excel there is one more file csv UTF-8

[AltruCoder]

I have run into encoding issue numerous times. It's generally not a bug with CsvHelper.

What encoding did your .txt file use? Most likely it used UTF-8.

Yes MS Excel can save the csv file as either ISO-8859-1 or UTF-8. If you save it in Excel with UTF-8 then it should work just fine with your code.

[AltruCoder]

I really wish MS Excel didn't give you the option to save as ISO-8859-1. Worse than that, saving as ISO-8859-1 shows as the default way to save a csv file. It should have been save as "CSV (Comma delimited)(*.csv)" for the UTF-8 version and save as "CSV ISO-8859-1 (Comma delimited)(*.csv)" for the ISO-8859-1 version.

[devanshudravid]

Yes @AltruCoder. Thanks for all the information.