Runtime: Assembly.LoadFrom throws 'already loaded' even if the assembly is not in memory

[raffaeler]

I am using 2.0.0-preview1-005977.
I have a Self Contained Deployment folder with all the needed dependencies to run my code.
The hosting part (exe) is in C++, so I load the CLR, create the AppDomain, load a first Assembly and everything works fine.

Later on, when C++ requests for it, I execute some .net code that execute an Assembly.LoadFrom(fullpathassemblyname.dll).
This causes the following exception:
System.IO.FileLoadException: 'Assembly with same name is already loaded'
But the assembly is not in memory. I can't see it either in VS Modules while debugging or even in process explorer.

The code I use to load the assembly is:
var assembly = Assembly.LoadFrom(fullname);
or
var assembly = AssemblyLoadContext.Default.LoadFromAssemblyPath(fullname);
For testing purposes only I also tried the following:
var assembly = Assembly.LoadFile(fullname);
and it works but it is not suitable as it loads just the given assembly and not the entire graph (if any).

So I have two questions:

• why Assembly.LoadFrom throws even if the given assembly is not in memory?
• if I remember well, the Assembly.LoadFrom of the .NET Framework does not throw even if the assembly was already loaded. Why did the behavior change?

[gkhanna79]

Is the assembly being loaded present on the TPA path you constructed for your host?

[raffaeler]

Yes it is, but given that I will allow others to 'drop' other assemblies in any folder, I would prefer not to make special cases for the assemblies in TPA.
Does this change the LoadFrom behavior? It never did AFAIR.

[gkhanna79]

.NET Core Binder/Loader are very different, by design, from Desktop Loader. Please see https://github.com/dotnet/coreclr/blob/master/Documentation/design-docs/assemblyloadcontext.md for details. Assembly.LoadFrom is equivalent to loading in DefaultContext.

We are running into a failure at https://github.com/dotnet/coreclr/blob/13e7c4368da664a8b50228b1a5ef01a660fbb2dd/src/vm/assemblynative.cpp#L230. It is possible that the message may not be the most appropriate but looks like we did run into an error loading the assembly.

CC @kouvel

[raffaeler]

I still don't get the problem.
Since I can it using Assembly.LoadFile, I know the assembly is good and the path is correct.
Also, even after failing with LoadFrom, the load succeeds if I invoke Type.GetType(full type name).

Even if the path is full qualified, it contains a "..\" segment. Do you believe this can be the problem? (I can't test until later today, GMT+1 time).

[jkotas]

Even if the path is full qualified, it contains a ".." segment. Do you believe this can be the problem?

Unlikely. The first thing done by LoadFrom is to normalize the path using Path.GetFullPath that will eliminate the ".." segment:

https://github.com/dotnet/coreclr/blob/master/src/mscorlib/src/System/Reflection/Assembly.CoreCLR.cs#L74

[raffaeler]

@jkotas I see.
Do you have any specific test I can make to better diagnose the problem?
For example I can run procmon to see what happens at the filesystem level.
Are there any CoreCLR environment vars to set to see a verbose tracer here?

Thanks

[jkotas]

I would run the process under C++ debugger (either attach as "native" in visual studio; or use windbg), stop on C++ exception and get the stacktrace where the C++ exception is thrown. The C++ part of the runtime is using C++ exceptions for error handling, so stopping on C++ exceptions tends to be a good way to find out where things went wrong.

[raffaeler]

@jkotas As my host is C++, it will be extremely simple to do later this night.
Are the symbols for the 2.0.0-preview1-005977 on the public MS server or should I clone the matching branch from the coreclr repo and build?

[jkotas]

Yes, the symbols should be published.

[raffaeler]

@jkotas the exception happens in this function:
LoadFromPath

This function throws if the dll exists in the TPA path but you try to load from a different path.
In other words I had this situation:

• The TPA contains the 'release' version of the dll
• LoadFrom tries to load the 'debug' version from a different path
  If I delete the file in the TPA or try to load the file inside the TPA, it works.
  (sorry, I initially believe the file was loaded from the TPA)

Questions:

• Is this a bug or what?
• I loaded the symbols but there is no source. Can I load the sources from github or what should I do to see them?

Thank you

[gkhanna79]

This is by design.

Thanks,
Gaurav

---

From: Raf [e-mail]
Sent: Thursday, July 13, 2017 12:56:46 PM
To: dotnet/coreclr
Cc: Gaurav Khanna; Comment
Subject: Re: [dotnet/coreclr] Assembly.LoadFrom throws 'already loaded' even if the assembly is not in memory (#12764)

@jkotashttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjkotas&data=02%7C01%7CGaurav.Khanna%40microsoft.com%7Cdb80201211f2487f27d708d4ca294b85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636355726092140787&sdata=QJjsAV1k0vHLWA7pYMvExlM3hh2ZDeglxx%2FnvHfbSKk%3D&reserved=0 the exception happens in this function:
LoadFromPathhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Fcoreclr%2Fblob%2F13e7c4368da664a8b50228b1a5ef01a660fbb2dd%2Fsrc%2Fvm%2Fassemblynative.cpp%23L250&data=02%7C01%7CGaurav.Khanna%40microsoft.com%7Cdb80201211f2487f27d708d4ca294b85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636355726092140787&sdata=snaZ8XS0UA5Q5lIXfFhvbz03zKh0J53drWgmFQgaLeI%3D&reserved=0

This function throws if the dll exists in the TPA path but you try to load from a different path.
In other words I had this situation:

• The TPA contains the 'release' version of the dll
• LoadFrom tries to load the 'debug' version from a different path
  If I delete the file in the TPA or try to load the file inside the TPA, it works.
  (sorry, I initially believe the file was loaded from the TPA)

Questions:

• Is this a bug or what?
• I loaded the symbols but there is no source. Can I load the sources from github or what should I do to see them?

Thank you

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Fcoreclr%2Fissues%2F12764%23issuecomment-315185153&data=02%7C01%7CGaurav.Khanna%40microsoft.com%7Cdb80201211f2487f27d708d4ca294b85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636355726092140787&sdata=62XBGgpSOel9gt72RDAWXHWqLhsqk4Nf5dXYyzR0ZeE%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJA3wfATOn8-TyEsTlPPYqYN5MEN1QW9ks5sNnZ-gaJpZM4OVdAw&data=02%7C01%7CGaurav.Khanna%40microsoft.com%7Cdb80201211f2487f27d708d4ca294b85%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636355726092140787&sdata=hiJ4yxdNATI1uBnkkjwMM7o6qOz8EEEi0VBqrAhPnj0%3D&reserved=0.

[raffaeler]

@gkhanna79 thanks to you.
Is there any chance to obtain a more specific message?
I was totally cheated from the message, otherwise it would have been easier to find the culcprit.

[jkotas]

Can I load the sources from github or what should I do to see them?

All .NET Core .dlls are stamped with git SHA1 hash they were built from (e.g. look for "Commit Hash" string in the binary). The VS debugger will eventually learn to fetch the sources automatically. Until that happens, you can get the matching sources by checking out the hash from github.

Is there any chance to obtain a more specific message?

Yes. Would you mind submitting PR to improve the message, or describe what you would like to see in the message?

[raffaeler]

@jkotas got it but the sources does not show up even if I enabled the source server support in the options.
The github path should already be in the pdb themselves, so what I am missing?

There are a couple of points where the same exception is thrown. I am trying to get the exact point with the sources and than I can submit the PR.

Thanks

[raffaeler]

@jkotas In other words, what is your GitLink setup to sync the public sources with this repo sources?
I tried in several ways with no luck at all...
Thank you

[jkotas]

The github path should already be in the pdb themselves, so what I am missing?

I do not think that the binaries are getting stamped properly for source server or gitlink to work. @dotnet/dnceng What is missing to get the source server or gitlink working for .NET Core binaries?

Here are steps that I have been using to get matching sources for given coreclr.dll (substitute c:\dotnet and c:\coreclr as appropriate):

c:\>powershell (Get-Item c:\dotnet\shared\Microsoft.NETCore.App\2.0.0-preview1-001978-00\coreclr.dll).VersionInfo.FileVersion
4.6.25212.04 built by: dlab-DDVSOWINAGE006. Commit Hash: 79727651f0cab62c46cb1fe34e12f2122d6d554c
c:\>cd coreclr & git checkout 79727651f0cab62c46cb1fe34e12f2122d6d554c

[raffaeler]

I read about the hash being in the FileVersion attribute, but without gitlink, the debugger will never know where to look for the sources, even if I download the whole repo with the matching hashes.
There must be some additional step I am now aware of...

[dleeapho]

@dagood, can you help with @jkotas question?

[dagood]

@jkotas Adding gitlink to the builds so that sources are indexed to GitHub is tracked by https://github.com/dotnet/core-eng/issues/499. It seems to me that we'll also need to get more serious about building portable PDBs, as a prereq: Windows builds are still making Windows PDBs right now. I'm not able to find a discussion (issue on CoreCLR or core-eng) about moving to Portable PDBs.

The external source server support we've had for some products is based on MyGet. If a symbol package has sources matching the PDBs, MyGet does some PDB rewriting and hosts the sources for us. I haven't seen CoreCLR put native code in the symbol packages. There's also the issue of Core-Setup redistributing the CoreCLR bits: I don't know what the MyGet behavior would be, and these sources might need to be flowed to Core-Setup too. Not sure this path is worth going down vs. gitlink to GitHub.

@raffaeler I haven't tried it for native debugging, but in my experience I get a dialog box that I can point to the current source file, then the debugger finds others based on that. With the repo checked out on disk at the right commit, I would think you can rig it up that way.

[jkotas]

@dagood This discussion is about gitlink support for C/C++ code. I believe that gitlink works for C/C++ fine, with classic non-portable PDBs. The portable PDBs should not be tied to this - there are no portable PDBs for C/C++.

[raffaeler]

@dagood the link you posted gave me a 404.
I tried putting a function breakpoint but VS2017.3 didn't ask me to find the source file, therefore I can't make it work. I guess I am missing something.
Also, if I have to browse the sources, what is gitlink useful for?

P.S. afaik gitlink works with old pdbs while sourcelink/sourcelinkv2 work with portable pdbs

[MattGal]

@raffaeler that's because it's a private repository.

[dagood]

@jkotas Noted, thanks.

@raffaeler Ah, I was thinking of the situation where you're already paused somewhere (e.g. at an exception). I'm not sure about setting a breakpoint first. To clarify about gitlink: in that section I was talking about a potential way to do this without gitlink to get where you need to go, because currently it's not in place.

[mjsabby]

Also commenting here for completeness: We should stick to the SourceLink specification (https://github.com/dotnet/core/blob/master/Documentation/diagnostics/source_link.md) regardless of what tool we end up using to stamp the pdb.

Specifically injecting something like the following into the PDB (got this from the source link spec):

{
"documents": {
"C:\src\CodeFormatter\": "https://raw.githubusercontent.com/dotnet/codeformatter/bcc51178e1a82fb2edaf47285f6e577989a7333f/"
}
}

[raffaeler]

@mjsabby I still can't understand if there is any way to get it work with the current bits.
I am trying with gitlink with no luck, but if you tell me that it works with sourcelink or any other way, it is fine for me.
Please note that I would need to make it work with both (or alternatively) native and managed code.

Thank you

[dagood]

@raffaeler The talk about SourceLink and GitLink is about what we can add to future builds to make the debugging process better. I'm not aware of anything that will help you set source breakpoints in your current 2.0.0-preview1 build.

[raffaeler]

Thank you @dagood. So basically when you want to debug, you clone the repo locally, then rebuild and than point your runtime to these binaries?
Or do you think that I can tweak the pdbs to force pointing to github (or a local clone)?

[dagood]

I haven't debugged CoreCLR myself, but building your own CoreCLR does sound like the most straightforward way to get PDBs pointing at your sources.

I believe MyGet repoints PDBs for their symbol package sources support, meaning it's possible, but I'm not familiar with tooling to do that.

[raffaeler]

@jkotas do you have any suggestion? What is your strategy? TIA

[jkotas]

When I want to debug, I checkout the sources that match the build: https://github.com/dotnet/coreclr/issues/12764#issuecomment-315631416 . No rebuilding required.