Extensions: Add Microsoft.Extensions.Logging.AzureEventHub

[jamesharling]

Is your feature request related to a problem? Please describe.

This is simply an additional logger provider for a popular choice of logging sink.

Describe the solution you'd like

I would like to be able to pump events raised through ILogger<T> to an additional logger provider, that sends the events to an Azure Event Hubs instance.

Describe alternatives you've considered

There are no alternatives within the M.E.L. ecosystem.

Additional context

The format of these events should be customisable, but a default formatter shall be provided.

[anurse]

Can you provide more context on why this needs to be in Microsoft.Extensions rather than just a third-party open-source package? By putting it here we sign up to maintain it in every release, so we need to be cautious about what we add as "official" providers. As you say, there is an M.E.L ecosystem so we strongly encourage the creation of additional third-party loggers.

This is simply an additional logger provider for a popular choice of logging sink.

Do you have some data to back up that this is a "popular choice of logging sink"? This is the first request I'm aware of for this logger, but I am admittedly not an expert here :).

cc @glennc @davidfowl @bradygaster

[jamesharling]

Sure. If it was a provider for a third-party sink (e.g. Seq) then I absolutely agree that the provider should be in a third-party library (maybe under aspnet-contrib or similar). However in this case it would be a Microsoft provider logging to a Microsoft Azure resource, so it seemed like a natural fit for inclusion in to Microsoft.Extensions.

In terms of the popularity of Azure Event Hubs as a destination for logging output, there is already lot of first-class support within the MS ecosystem for routing logs to it, such as for Azure API Management, Azure Diagnostic Logs and Azure SQL Audit Logs. There are also many adapters and plugins for other logging libraries, such as Serilog, NLog and log4net.

[anurse]

It certainly makes a lot of sense as a logger (I notice Serilog also has it which means there is a path to integrate M.E.L with Azure Event Hubs via Serilog). My concern is around making this a first-party library in this repo. We've generally avoided major integrations with external components in our first-party loggers (Console, etc.) with the small exception of Azure App Service (which has been a challenge).

Basically what I'm saying is I don't expect us to have the capacity to take this on as a first-party library any time soon (i.e. in the 3.0 timeframe). I'm not sure I agree that this wouldn't fit as an aspnet-contrib project. For example, a lot of assorted Authentication Providers are in that project rather than under the Microsoft. root. This doesn't seem that different to me.

To be totally realistic, starting up a project (on aspnet-contrib or yourself) is likely to get this out to people much quicker than if you want us to consider shipping this first-party. We won't be able to look in depth at this until .NET Core 3.0 gets out the door, so this will likely sit idle for a while. Getting a third-party package up and running is a great way to prove the value of this, and we'd certainly be happy to link it from resources like our docs on Logging.

[jamesharling]

In that case, would there perhaps be scope to start it as a third-party package with it being considered for inclusion in M.E.L. after 3.0 ships, e.g. for 3.1?

[anurse]

Yeah, that seems like a reasonable way forward here. I'd challenge slightly the assumption that these things need to be first-party. In this case, since it's integrating Microsoft libraries with Azure tools, it does make some sense. However, in general we'd love to have a broad third-party community for these components. It makes the open-source ecosystem around .NET better.

[anurse]

We're generally happy to help review code, even for these third-party components, so feel free to tag me in PRs to wherever this lands (and I can bring in other folks from the team). While we aren't able to commit to the full first-party support level, we really want to help the community build up around these providers and code reviewing is one concrete way we can do that.

[jamesharling]

Sounds like a good plan. I will create a third-party library for this for now, and tag you in the PRs for review - that would be really useful.

How is best to track this issue for delivery in 3.1?

[anurse]

We don't have plans for significant features in 3.1 as it is planned to be a small release focused on bugfixes. I'll put this issue in the backlog to consider as we get more feedback.

[jamesharling]

Now 3.1 has been released, bumping this issue. @anurse

[msftbot[bot]]

As part of the migration of components from dotnet/extensions to dotnet/runtime (https://github.com/aspnet/Announcements/issues/411) we will be bulk closing some of the older issues. If you are still interested in having this issue addressed, just comment and the issue will be automatically reactivated (even if you aren't the author). When you do that, I'll page the team to come take a look. If you've moved on or workaround the issue and no longer need this change, just ignore this and the issue will be closed in 7 days.

If you know that the issue affects a package that has moved to a different repo, please consider re-opening the issue in that repo. If you're unsure, that's OK, someone from the team can help!

[jamesharling]

Bump to keep issue open

[msftbot[bot]]

Paging @dotnet/extensions-migration ! This issue has been revived from staleness. Please take a look and route to the appropriate repository.

[ericstj]

@pakrym @Petermarcu is this another candidate to be owned by Azure SDK?

[pakrym]

Maybe. We would have to figure out how popular EventHubs are as the log destination.

Considering that this is the first request of it's kind I'm slightly reluctant to ship a first party implementation just yet.

Feel free to move the issue into azure-sdk-for-net for tracking and collecting requests though.

[ericstj]

Closed in favor of https://github.com/Azure/azure-sdk-for-net/issues/12033