Machinelearning: Make DataFrame ReadCsv more efficient

[pgovind]

DataFrame.ReadCsv at the moment loops over all the rows in a file twice: once to find the number of rows and schema, and the 2nd time to append the data. Each line is read in as a string and string.Split is called on it. Profiling shows that a sizeable chunk of time is spend in the Split calls. We should explore using a ReadOnlySpan to parse the file and avoid all the intermediate strings.

[MarcoRossignoli]

Is there a reason to not make api ReadStream async?

[pgovind]

Nope. This is not the most performant implementation. This exists at the moment so folks can work on real data. Making this method more efficient is on my backlog.

[avinds]

Could we also make it such that it can handle escaping using quoted strings.