Azure-sdk-for-net: [BUG] EventProcessorHost TypeLoadException

[SteffenMangold]

Describe the bug
A TypeLoadException is thrown for 'Microsoft.WindowsAzure.Storage.CloudStorageAccount' when using EventProcessorHost.

Exception or Stack Trace

           at Microsoft.Azure.EventHubs.Processor.AzureStorageCheckpointLeaseManager..ctor(String storageConnectionString, String leaseContainerName, String storageBlobPrefix)

at Microsoft.Azure.EventHubs.Processor.EventProcessorHost..ctor(String eventHubPath, String consumerGroupName, String eventHubConnectionString, String storageConnectionString, String leaseContainerName)
at Solytic.Services.DataReadingHostService.Program.Main(String[] args) in C:UsersSteffenMangoldsourcereposSolytic 2.0 Backend ServicesSolytic.Services.DataReadingHostServiceProgram.cs:line 16

To Reproduce

        <Project Sdk="Microsoft.NET.Sdk">
          <PropertyGroup>
            <OutputType>Exe</OutputType>
            <TargetFramework>netcoreapp2.2</TargetFramework>
            <DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
          </PropertyGroup>

          <ItemGroup>
            <PackageReference Include="Microsoft.Azure.EventHubs" Version="4.0.0" />
            <PackageReference Include="Microsoft.Azure.EventHubs.Processor" Version="4.0.0" />
            <PackageReference Include="Microsoft.Azure.Storage.Blob" Version="11.0.0" />
            <PackageReference Include="Microsoft.Azure.Storage.Common" Version="11.0.0" />                
            <PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.9.2" />
          </ItemGroup>
        </Project>

Code Snippet

        EventProcessorHost eventProcessorHost = new EventProcessorHost("test"
            , "Event Hubs"
            , "Consumer1"
            , AzureEventHubsModel.GetDefaultConnectionString()
            , string.Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}", "lease", "QZPDA8ZOTW1ndnTArwqSFlflnR3P8vOdCifSj5sEEJP/MGEM1fmYOsJRKgZWZuaSClOyobTunGT8kqDAMWRjVg=="))
        {
            PartitionManagerOptions = new PartitionManagerOptions()
            {
                RenewInterval = TimeSpan.FromSeconds(10),
                LeaseDuration = TimeSpan.FromSeconds(60)
            }
        };

Setup (please complete the following information):

• OS: Windows 10
• IDE : VS2019

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [ x] Bug Description Added
• [ x] Repro Steps Added
• [ x] Setup information Added

[SteffenMangold]

~Downgrade "Microsoft.Azure.EventHubs.Processor" to 3.0.0 solves the problem.~
Helps only with IntelliSens error, still not usable. (See above for details)

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

[SteffenMangold]

This is the source of the problem.

[SteffenMangold]

It looks like the problem is related to the "Microsoft.Azure.Storage.Blob" library.
NETStandard and NETCore version is using a different namespace.

But I can't patch it because n the current branch there is no CloudStorageAccount class!?
I don't get it. :(

[SteffenMangold]

Any news on this? Because in relation with NetCore and "Microsoft.Azure.EventHubs" the EventProcessorHost is not useable in any version.

Version 3.0.0 throws:
Method not found: 'Microsoft.Azure.EventHubs.EventHubClient Microsoft.Azure.EventHubs.EventHubClient.Create(System.Uri, System.String, Microsoft.Azure.EventHubs.ITokenProvider, System.Nullable'1<System.TimeSpan>, Microsoft.Azure.EventHubs.TransportType)'.

Stack:
at Microsoft.Azure.EventHubs.Processor.EventProcessorHost.CreateEventHubClient() at Microsoft.Azure.EventHubs.Processor.PartitionManager.<GetPartitionIdsAsync>d__6.MoveNext()

Currently, we can't develop any further! @jfggdl

[SteffenMangold]

This is a random post - I have 3.0.0 working (had same issue as you with 4.0.0)
When going from 2.?.? -> 3.0.0 I had issue with:

// Azure/azure-event-hubs-dotnet#383
// hostName: Program.ServiceInstanceName,
hostName: Program.ServiceInstanceName + "_" + DateTimeOffset.Now.ToDateTimeString().Replace(" ", "_"),

I think azure uses GUID:s for hostname (or that is their test case) - but my memory might be corrupted as its to long ago

Can you share what version of Microsoft.Azure.EventHubs you are using and what overload of EventProcessorHost constructor? Because I wonder why I can't run 3.0.0!?

[larslarsson]

That problem doesn't seem to exist anymore, should have tested.
Ah you run .net core, hade exact same error on .net framework

<package id="Microsoft.Azure.EventHubs" version="3.0.0" targetFramework="net462" />
<package id="Microsoft.Azure.EventHubs.Processor" version="3.0.0" targetFramework="net462" />

I used IEventProcessorFactory but I do not remember why nor do I remember the documentation anymore :(

                var listenerEventHubFactory = new Models.Listeners.ListenerEventHubProcessFactory(listenerMulti, eventHubCfg);
                var host = new Microsoft.Azure.EventHubs.Processor.EventProcessorHost(
                    hostName: Program.ServiceInstanceName + "_" + DateTimeOffset.Now.ToDateTimeString().Replace(" ", "_"),
                    eventHubPath: EventHubSettings.Default.HubName,
                    consumerGroupName: EventHubSettings.Default.ConsumerGroup,
                    eventHubConnectionString: EventHubSettings.Default.Connection,
                    storageConnectionString: EventHubSettings.Default.StorageConnection,
                    leaseContainerName: EventHubSettings.Default.HubName
                );
                listenerStartables.Add(new Data.DelegateStartable(() => host.RegisterEventProcessorFactoryAsync(listenerEventHubFactory, options)));

[SteffenMangold]

@larslarsson ~but is your core application Net standard or Net core?
Also I think its important if you already using Microsoft.Azure.Storage.Blob or the old WindowsAzure.Storage library. I think the dependency of both somehow not work beside.~

Ok! get it working by also reverting Microsoft.Azure.EventHubs to 3.0.0.

[larslarsson]

Good for you - and yes I was wrong in that I use .net framework as base not .net core or .net standard.

Was thinking I had to go to my shame corner & cry since shelvset worked after rebuild but closing VS2019 & reopen it & rebuilding gives me the exception again.

.net framework 4.6.2
2019-08-21 17:53:00,009 [1 ] FATAL (Program) - UnhandledExceptionEventArgs: at Microsoft.Azure.EventHubs.Processor.AzureStorageCheckpointLeaseManager..ctor(String storageConnectionString, String leaseContainerName, String storageBlobPrefix)
at Microsoft.Azure.EventHubs.Processor.EventProcessorHost..ctor(String hostName, String eventHubPath, String consumerGroupName, String eventHubConnectionString, String storageConnectionString, String leaseContainerName, String storageBlobPrefix)

2019-08-21 17:53:00,011 [1 ] FATAL (Program) - UnhandledExceptionEventArgs, message:Could not load type 'Microsoft.WindowsAzure.Storage.CloudStorageAccount' from assembly 'Microsoft.Azure.Storage.Common, Version=11.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

[axisc]

@SteffenMangold

Thanks for raising this issue. We identified that one of the libraries we depend modified the contract.

We will update the dependency in the next release (4.1), which should solve the problem on your end.

I'll keep this issue open until then.

[serkantkaraca]

The failure was due to storage namespace change from 9.4 to 11.0. Issue fixed with EPH 4.1.0 release. Please reactivate if needed.