Csvhelper: ClassBuilder, runtime Model from CSV file

[papyr]

Hi this is great, can you please support a classbuilder to create a class from CSV headers at runtime.

//build dynamic class
public List<TModel> ReadCsvFile<TModel, TModelMapper>(string file) where TModelMapper : ClassMap<TModel>
{
    var configuration = new Configuration
    {
        IncludePrivateMembers = false,
        IgnoreBlankLines = true,
        Delimiter = ";" // You may want to change this
    };
    configuration.RegisterClassMap<TModelMapper>();
    using (var csvFile = File.OpenText(file))
    {
        var parser = new CsvHelper.CsvReader(csvFile, configuration);
        return parser.GetRecords<TModel>().ToList();
    }
}


// Then you reading the file like this
//************************************// ???
// Question -- how do I REPLACE the `CustomerModel` part to be REFLECTED extracted or name after the CSV filename???
var customers = ReadCsvFile<CustomerModel, CustomerModelMapper>(@"customers.csv");

[JoshClose]

You want to know how to create CustomerModelMapper that you're passing into ReadCsvFile? If that's the case, just don't register a class map at all. The library will generate one on the fly based on the header names of the file.

[papyr]

Hi @JoshClose thanks for the response.

I am asking to
Step 1) create the actual C# Class, from the CSV file headers - so I can write/save the xxx.cs class to disk. I dont know what headers will be in the CSV file so I need to build the class dynamically from the headers.
then Step 2) create the model mapper and fill in. inside file helpers they a DynamicClassBuilder Object.

I added code to show we can build C# class.

[JoshClose]

Oh... Will all the properties be strings then?

[weedkiller]

For now stringsure, that way we can get it going and improve. I would prefer to see if if we can try the casting/reflection to find the correct type - if it fails, stringtype it is!

[JoshClose]

If you don't know the field types or the property names ahead of time, how are you going to use the class later?

[papyr]

@JoshClose you right about the not knowing the types, its a catch 22 really! So, I was looking to just make them string types. Later I was going to use a try cast as to see if it succeeded, but still risky. So string type is the best to get it going.

[weedkiller]

hi polite bump, any chance i can get a sample /snippet to do this?

[JoshClose]

I don't understand how you'll be able to access the properties later if you have no clue what the names of the properties are. It would probably be better to use a dictionary instead, where the keys are the names of the headers. That way you can easily get the names.

void Main()
{
    var s = new StringBuilder();
    s.AppendLine("Id,Name");
    s.AppendLine("1,one");
    s.AppendLine("2,two");
    using (var reader = new StringReader(s.ToString()))
    using (var csv = new CsvReader(reader))
    {
        var records = new List<IDictionary<string, object>>();
        csv.Read();
        csv.ReadHeader();
        while (csv.Read())
        {   
            var record = new Dictionary<string, object>();
            records.Add(record);
            foreach (var header in csv.Context.HeaderRecord)
            {
                record[header] = csv.GetField(header);
            }                       
        }

        records.Dump();
    }
}

[papyr]

Thats fine for now we can have the headers as string col types.

But how can I use CsvHelper to emit the Class definition (write the file to physical disk using the filename.cs , and IQueriableClass)

        csv.Read();
  csv.ReadHeader();

in the above how can I export the C# class. csv.DynamicBuilder.Class.ExportToDisk(string fileName) & Iqueryable listOfSomething = csv.DynamicBuilder.Class.IQuerable<T>

[JoshClose]

You could use dynamic then.

void Main()
{
    var s = new StringBuilder();
    s.AppendLine("Id,Name");
    s.AppendLine("1,one");
    s.AppendLine("2,two");
    using (var reader = new StringReader(s.ToString()))
    using (var writer = new StringWriter())
    using (var csvReader = new CsvReader(reader))
    using (var csvWriter = new CsvWriter(writer))
    {
        dynamic records = csvReader.GetRecords<dynamic>();

        csvWriter.WriteRecords(records);

        writer.ToString().Dump();
    }
}

Output:

Id,Name
1,one
2,two

[papyr]

Hi @JoshClose I want to get/create the class definition.
for e.g. output for csv.DynamicBuilder.Class.IQuerable

public class FileName : IQueryable {
Type Colname;
Type Colname2;
}

[JoshClose]

Creating a type on the fly isn't built in. Creating instances from a type definition is though. I don't really see a reason to add creating types to the library either.

Here is how you can do it though.

void Main()
{
    var s = new StringBuilder();
    s.AppendLine("Id,Name");
    s.AppendLine("1,one");
    s.AppendLine("2,two");
    using (var reader = new StringReader(s.ToString()))
    using (var csv = new CsvReader(reader))
    {
        csv.Read();
        csv.ReadHeader();
        var type = CreateType(csv.Context.HeaderRecord);
        var records = csv.GetRecords(type);
        records.Dump();
    }
}

public Type CreateType(string[] headers)
{
    var assemblyName = new AssemblyName($"DynamicAssembly_{Guid.NewGuid():N}");
    var assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.RunAndSave);
    var moduleBuilder = assemblyBuilder.DefineDynamicModule("DynamicModule");
    var typeBuilder = moduleBuilder.DefineType($"DynamicType_{Guid.NewGuid():N}", TypeAttributes.Public);

    foreach (var header in headers)
    {
        var fieldBuilder = typeBuilder.DefineField($"_{header}", typeof(string), FieldAttributes.Private);
        var propertyBuilder = typeBuilder.DefineProperty(header, System.Reflection.PropertyAttributes.HasDefault, typeof(string), Type.EmptyTypes);

        var getMethodBuilder = typeBuilder.DefineMethod($"get_{header}", MethodAttributes.Public, typeof(string), Type.EmptyTypes);
        var getIlGenerator = getMethodBuilder.GetILGenerator();
        getIlGenerator.Emit(OpCodes.Ldarg_0);
        getIlGenerator.Emit(OpCodes.Ldfld, fieldBuilder);
        getIlGenerator.Emit(OpCodes.Ret);

        var setMethodBuilder = typeBuilder.DefineMethod($"set_{header}", MethodAttributes.Public, null, new Type[] { typeof(string) });
        var setIlGeneator = setMethodBuilder.GetILGenerator();
        setIlGeneator.Emit(OpCodes.Ldarg_0);
        setIlGeneator.Emit(OpCodes.Ldarg_1);
        setIlGeneator.Emit(OpCodes.Stfld, fieldBuilder);
        setIlGeneator.Emit(OpCodes.Ret);

        propertyBuilder.SetGetMethod(getMethodBuilder);
        propertyBuilder.SetSetMethod(setMethodBuilder);
    }

    var type = typeBuilder.CreateType();

    return type;
}

[papyr]

@JoshClose @weedkiller this will help us get Enumerable from CSV as well. See this link..

private IEnumerable<string[]> CsvToEnumerable(string fileName)
{
    char[] separator = new[] { ',' };
    string currentLine;

    using (var stream = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
    using (var reader = new StreamReader(stream, Encoding.Default, true, 1024))
    {
        while ((currentLine = reader.ReadLine()) != null)
        {
            yield return currentLine.Split(separator, StringSplitOptions.None);
        }
    }
}

Ultimately CsvHelper is a versatile toolbelt for CSV's, so trying to get a class from CsvHelper would be very helpful, if you look at the other popular opensource FH, they have a DynamicClassBuilder as well

[JoshClose]

There is really no use for it in the CsvHelper library though.