Confluent-kafka-dotnet: avrogen with multile avro schema files

[alfhv]

Description

I woule like to use multiple avro files when generating C# classes with avrogen.
Looking at apache.Avro code at: https://github.com/apache/avro/blob/e3e41dcd27822ac3fd4692291d1bbea4f8a294dc/lang/csharp/src/apache/main/Schema/Schema.cs#L210

when reading multipe files it is not possible to reuse previous loaded schema because "new SchemaNames()" is passed.

There is a way to use multipe avro files ?

How to reproduce

Checklist

Please provide the following information:

• [ ] A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
• [ latest] Confluent.Kafka nuget version.
• [ latest] Apache Kafka version.
• [ ] Client configuration.
• [ ] Operating system.
• [ ] Provide logs (with "debug" : "..." as necessary in configuration).
• [ ] Provide broker log excerpts.
• [ ] Critical issue.

[mhowlett]

you'll need to use GenericRecord. yep, I agree this API is less than ideal.

[alfhv]

I did some changes in avrogen, available here https://github.com/alfhv/avro, to make it work on my use case.
main changes are:

• allow extra parameter to specify multiple schema files (i use a list of files but it could be a directory) the file order is important, at least in this very early version, to load types references in correct order.
• add new method in AvroGen class:
  static void GenSchema(List<string> infiles, string outdir, IEnumerable<KeyValuePair<string, string>> namespaceMapping)
• add new method in Schema.cs class:
  public static Schema Parse(string json, SchemaNames schemaNames)

it could be possible to allow in some way this functionality with avrogen tool ??

Actually, the current API should be more open so we can overload some methods and code generation behavior, my use case is wider than just use multiple schema files.
ex: I have a base schema file (to inherit types from it) and then other schemas files from where I want to generate C# classes with avrogen but ignoring (not generating) classes from base schema file. I have already implemented in some way this behavior in my own version of avrogen but I would like to stay close to original Apache version in some way.

[gustavoferrazfontes]

@alfhv
your changes helped me a lot! Thank you and congratulations for your initiative.