Go: plugin: add Windows support

[QuestionPython]

hi
Plugin Pkg Work for Windows!?
i want use this for mac,linux,win,... os.
when(what time) fix this?

https://golang.org/pkg/plugin/

[bradfitz]

There is currently nobody working on it, as far as I know.

/cc @alexbrainman @crawshaw

[QuestionPython]

mean in go 1.8 , plugin pkg work for apple-mac,windows and more ?

[bradfitz]

@questionpython, yes, it's even documented in multiple places:

https://golang.org/pkg/plugin/

Currently plugins only work on Linux.

https://golang.org/doc/go1.8#plugin

Plugin support is currently only available on Linux.

[alexbrainman]

There is currently nobody working on it, as far as I know.

I am not working on it. Sorry.

Alex

[bradfitz]

We delete all "me too" voting comments per https://golang.org/wiki/NoMeToo. Vote with emoji reactions at top instead.

[bradfitz]

The "me too" comments wouldn't stop, so this issue is now locked. If there are updates, they will be made here.

[0xdevalias]

Out of curiosity, do we know how much effort it would take to implement windows support? Or if there are any blockers to it (and what they are?)

[akavel]

Notably, the recently published Go kernel for Jupyter notebooks is using buildmode=shared, and thus doesn't currently support Windows natively. This is a very cool use case, adding a REPL-like live coding feature to the Go ecosystem, thus it would be really awesome if someone tried to start work on buildmode=shared on Windows to support this use case.

Similar to @0xdevalias , I'm quite interested in some hints as to what is missing for this to work on Windows? I'm especially curious what extra work is needed given that c-shared is already implemented on Windows?

[alexbrainman]

@0xdevalias and @akavel I don't have any effort estimation or any hints as to what missing here. I have not actually looked at what is involved. I am so much behind at fixing old issues ...

Alex

[akavel]

@alexbrainman Thanks! I'll ask on golang-dev then, maybe someone else can shed some light (edit: link to the thread)

[0xdevalias]

There doesn't seem to be a huge amount to it in the src: https://github.com/golang/go/tree/master/src/plugin

My completely naive guess would be figuring the windows equivalents to the C-bindings in plugin_dlopen.go.

The main functions I can see there are:

• https://linux.die.net/man/3/dlopen : The function dlopen() loads the dynamic library file named by the null-terminated string filename and returns an opaque "handle" for the dynamic library.
• https://linux.die.net/man/3/dlsym : The function dlsym() takes a "handle" of a dynamic library returned by dlopen() and the null-terminated symbol name, returning the address where that symbol is loaded into memory.

Googling for "dlopen equivalent windows" led me to the following:

• https://github.com/alainfrisch/flexdll : an implementation of a dlopen-like API for Windows
  
  
  
  • "FlexDLL implements mostly the usual dlopen POSIX API, without trying to be fully conformant though (e.g. it does not respect the official priority ordering for symbol resolution). This should make it easy to port applications developped for Unix."
  
  
• LoadLibrary: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx

And "dlsym equivalent windows":

• https://stackoverflow.com/questions/4184017/dlopen-dlsym-on-the-main-executable-how-portable-is-it
  
  
  
  • "Windows uses LoadLibrary() and GetProcAddress() instead."
  
  
• GetProcAddress: https://msdn.microsoft.com/en-us/library/windows/desktop/ms683212(v=vs.85).aspx
• https://www.symantec.com/connect/articles/dynamic-linking-linux-and-windows-part-two
  
  
  
  • "Both Linux and Windows provide routines (such as dlopen() and dlsym() in Linux, and LoadLibrary() ,GetProcAddress() in Windows)"
  
  
• https://www.experts-exchange.com/questions/22762973/Want-to-create-a-wrapper-class-for-dlopen-dlsym-dlclose-to-work-on-both-UNIX-and-windows-platform.html
  
  
  
  • "On windows, I believe LoadLibrary() == dlopen(), GetProcAddress() == dlsym(), and FreeLibrary() == dlclose()"
  
  

So from that, it sounds like we have the following premise to work from:

• dlopen in *nix roughly maps to LoadLibrary in windows
• dlsym in *nix roughly maps to GetProcAddress in windows

The main definitions in FlexDLL don't look too complex.. but there is quite a bit of extra code around those that may be required too:

• https://github.com/alainfrisch/flexdll/blob/master/flexdll.c#L83-L117

Hopefully this helps scope out what will be required/start pointing in the right direction :)

[jclc]

All the posts seem to be concerned with the loading of symbols, but does the compiler support producing a plugin (presumably DLL) on Windows?

[alexbrainman]

does the compiler support producing a plugin (presumably DLL) on Windows?

It is possible to build Windows DLL from your Go code. You want -buildmode=c-shared 'go build' flag for that. See #26714 for an example. 'go build' command uses gcc under covers to build DLL.

Alex

[jclc]

I've been hacking on this issue for a while and it seems to be going well for now. I've managed to load a dll built with -buildmode=c-shared and call its init function. The only limitation of this is that the plugin needs to have a main function or it won't compile. I'm developing on Linux using GCC and Wine. Just a few questions if anyone could clarify:

What exactly is going on in this function? The dlopen implementation calls this function and apparently returns the symbols; it doesn't work with Windows's shared objects.
https://github.com/golang/go/blob/master/src/runtime/plugin.go#L10

Secondly, I couldn't find any consistent guidelines for using wide strings with CGO so I ended up depending on unicode/utf16 and unicode/utf8. However, go/build/deps_test.go has pretty strict restrictions on which packages you can import. Is this a problem?

Edit: I guess this isn't so straightforward as I thought. -buildmode=plugin adds some metadata that is needed to find the exported symbols. Reading the PE data (using pev's readpe) doesn't show any of the functions that the plugin is meant to export, only init and main. When loading it, the init function is run implicitly.

[bradfitz]

@jclc, deps_test.go is largely there to force us to think about dependencies and decide whether they're acceptable. An edge from plugin to unicode/utf* seems fine, especially since syscall can already use those, and plugin => to syscall is already permitted.

[heaths]

@jclc several toolchains will simply add a DllMain for you automatically. I recommend a similar model here where the compiler (with modifications as necessary) does that automatically. There should be no need for a plugin developer to define this function or make modifications to it (it's pretty rare to even in production).

As this is a blocking issue to something I'm trying to get into Moby, I'd be willing to help if you need it.

[ImVexed]

I've been hacking on this issue as well as far as getting https://github.com/golang/go/blob/master/src/plugin/plugin_dlopen.go functioning on windows. As @0xdevalias gave insight into, LoadLibrary & GetProcAddress are the two equivalent we're looking for.

As far as @heaths suggestion to automatically generate a DllMain, I think this may be our best option. While we could use something similar to how https://github.com/alainfrisch/flexdll/blob/master/flexdll.c#L83-L117 does it by using either DONT_RESOLVE_DLL_REFERENCES or LOAD_LIBRARY_AS_DATAFILE in LoadLibraryEx I think it may just cause more work as on top of not invoking DllMain it also does not load any references to dependencies and would be quite tedious to implement manually.

[jclc]

There shouldn't be a need to use cgo, since package syscall has functions for loading symbols on Windows, NewLazyDLL() and (*LazyDLL).Handle(). I missed these initially since they don't show up on godoc. There's also helper functions for Windows wide strings.

[heaths]

@ImVexed LOAD_LIBRARY_AS_DATAFILE also doesn't do other things like rebase addresses as necessary. As the documentation reads,

Nothing is done to execute or prepare to execute the mapped file. Therefore, you cannot call functions like GetModuleFileName, GetModuleHandle or GetProcAddress with this DLL.

[ImVexed]

@jclc Interesting, do we know if there are any reasons as to why NewLazyDLL isn't used currently for plugins?

[jclc]

I'm not an expert, but I believe c-shared Go libraries ship their own Go runtime along with garbage collection. I don't think you'd want that if you're loading it for use in a Go program. Also, see my earlier post. plugin_dlopen calls https://github.com/golang/go/blob/master/src/runtime/plugin.go#L10 to hook up with the module somehow.

[alexbrainman]

do we know if there are any reasons as to why NewLazyDLL isn't used currently for plugins?

Just a guess. Plugins where developed by Linux and Darwin developers. There is no syscall.NewLazyDLL on Linux or Darwin.

Alex

[cchamplin]

@heaths, @ImVexed It shouldn't be necessary to do strange things around generating the DLL entry point, GCC takes care of the wiring up for that mostly implicitly. If you adjust go to export go.link.addmoduledata in on windows as an external you can you use the '-e' flag when linking with gcc to have it call into the right initialization function for plugins. The underlying init calls that LdrpCallInitRoutine makes and the related CRT_INIT are all handled by GCC.

The bigger issue seems to be with relocations and the GOT/PLT. With some small modifications it seems like I can get .TEXT symbols looking okay, but the stuff in .DATA is seems to be getting clobbered by GCC (specifically local.moduledata is trashed, but it looks like that's just the start of the structure data that gets completely hosed). I suspect that go is outputting these somewhere or in such a way that GCC is co-mingling the data and stuff either isn't getting relocated correctly or tables are getting partially overwritten/corrupted.

If there is someone who has good knowledge the PE format and by some miracle with GCC-Windows as well that would probably be extremely helpful here, otherwise I'll keep trying to muddle through.

[akavel]

@cchamplin Not claiming to have answers, and not sure if that might be in any way helpful, but some years ago I managed to emit some PE/COFF .o files with gcc-accepted .RSRC sections in akavel/rsrc; I faintly recall having to write relocations for it, so it may or may not help you in some way. I remember that my approach was generally to try to emit something based on what I understood from various resources on the Internet, then try to build identical .o with GCC's windres. Inevitably there were some differences, so then I opened both files in some hex editor and tried to understand what's wrong, and what's irrelevant (dates, etc.; and more often than I'd want, if couldn't understand, copy blatantly... :/) Rinse and repeat until I got I think a byte-perfect copy, which would thus pass gc/gcc linkage. Maybe a similar approach could be possible with c-shared? E.g. based on some minimal .s file compiled with gc vs. an ugly twin in gcc? Not sure if that's at all possible, or makes sense (esp. given the whole Go runtime needing to be linked). I'm afraid I might not remember anything about PE anymore now; just wanted to say, that if by any chance this could be helpful to you in any way, I seem to be calculating some relocations in ~coff.go:387 (though maybe somewhere else too?). I seem to remember that the "formula" for calculating relocations seemed to be described in a somewhat confusing/misleading way on MSDN, at least for me; when I finally got it to work I believe I felt happy and satisfied, but also annoyed and cheated.

Anyway, keeping fingers crossed for you super hard! Binary hacking on Windows certainly requires stalwart dedication :)

[cchamplin]

@akavel I think anything helps at this point :)

So here is what I believe is happening. There are some symbols local.moduledata that are supposed to end up in .noptrdata. At somepoint (I think maybe gcc is doing it, but it could be pe.go) the stuff is supposed to be in .noptrdata is getting stuffed into .data (which by itself isn't necessarily a bad thing, depending on other things like whether or not the garbage collector sees this other data). During this time the symbols in relation to the actual data structures that are there are getting offset (I don't know why or how). So when local.addmoduledata tries to grab the address for local.moduledata it's getting an address that has a bunch of the utf8 static data in it. I'm wondering if this is a relocation problem but I'm not sure...obviously this isn't happening in ELF partially because .noptrdata doesn't get folded into .data. I also can't actually find out where in the linker the symbol binary data is written out (I can see where the symbols themselves are written in pe.go but not sym.P) so if someone knows where that is it might be helpful.

[alexbrainman]

There are some symbols local.moduledata that are supposed to end up in .noptrdata. At somepoint (I think maybe gcc is doing it, but it could be pe.go) the stuff is supposed to be in .noptrdata is getting stuffed into .data

Are you talking about pe sections named .noptrdata and .data ? Because I do not remember that Go linker generates .noptrdata pe section in the object file to be used for external linker. As far as I remember whatever symbol named as .noptrdata endup in either .text, .rdata or .data pe section. But, regardless, relocations should be correct, because they apply to the whole pe section. Maybe Go linker does not align some symbols or sections properly.

It is hard for me to advise here. If you have some repro for me to play with, I might be able to help.

Alex

[cchamplin]

As far as I remember whatever symbol named as .noptrdata endup in either .text, .rdata or .data pe section.

Okay that's good to know, I wasn't sure if it was gcc doing it or the go linker

It is hard for me to advise here. If you have some repro for me to play with, I might be able to help.

Understood, let me see if I get pull out a minimum set of changes needed to get underway and get them in a repo

[alexbrainman]

let me see if I get pull out a minimum set of changes needed to get underway and get them in a repo

SGTM. Thank you.

Alex

[cchamplin]

@alexbrainman
Branch with minimal set of changes can be found here https://github.com/cchamplin/go/tree/plugins_windows
Test Project Here
https://github.com/cchamplin/plugin_test

[cchamplin]

Okay some additional follow up, it actually looks like the output assembly and symbols are fine there may not be corruption, I think what I was seeing was slight differences in how the memory is represented in ELF vs PE, but doing more thorough checking of the memory at runtime things might be okay.

What appears to be happening is that in a functional example (*nix)
runtime.addmoduledata loads lastmoduledatap which I guess should have a pointer to firstmoduledata.

Whats happening on windows is that lastmoduledatap has a pointer to local.moduledata which is apparently not correct.

I'm not sure where lastmoduledatap is getting set to firstmoduledata as I can't seem to find it in code. The code I've found shows that it should have the address of local.moduledata. symtab.go:680

[mattn]

As far as I can see, most of C code in plugin_windows.go can be replaced to Go.

[cchamplin]

@mattn This is correct, it's only the way it is to try and get other parts working.

[alexbrainman]

What appears to be happening is that in a functional example (*nix)
runtime.addmoduledata loads lastmoduledatap which I guess should have a pointer to firstmoduledata.

I see about the same. I could not stop gdb in the DLL loading code - I do not know how to do it. I tried breakpoint and rwatch. But I used objdump -d, and I can see this

000000006fdf7f40 <runtime.addmoduledata>:
    6fdf7f40:   41 57                   push   %r15
    6fdf7f42:   4c 8b 3d af 21 00 00    mov    0x21af(%rip),%r15        # 6fdfa0f8 <runtime.lastmoduledatap>
    6fdf7f49:   49 8b 07                mov    (%r15),%rax
    6fdf7f4c:   48 89 b8 c0 01 00 00    mov    %rdi,0x1c0(%rax)
    6fdf7f53:   4c 8b 3d 9e 21 00 00    mov    0x219e(%rip),%r15        # 6fdfa0f8 <runtime.lastmoduledatap>
    6fdf7f5a:   49 89 3f                mov    %rdi,(%r15)
    6fdf7f5d:   41 5f                   pop    %r15
    6fdf7f5f:   c3                      retq   

000000006fdf7f60 <go.link.addmoduledata>:
    6fdf7f60:   48 8d 3d b9 71 00 00    lea    0x71b9(%rip),%rdi        # 6fdff120 <local.moduledata>
    6fdf7f67:   e8 d4 ff ff ff          callq  6fdf7f40 <runtime.addmoduledata>
    6fdf7f6c:   c3                      retq   
    6fdf7f6d:   cc                      int3   
    6fdf7f6e:   cc                      int3   
    6fdf7f6f:   cc                      int3   

but there is nothing at 0x6fdfa0f8 address in objdump output.

I'm not sure where lastmoduledatap is getting set to firstmoduledata as I can't seem to find it in code. The code I've found shows that it should have the address of local.moduledata. symtab.go:680

I am not familiar with this code. Maybe @ianlancetaylor can help.

But, regardless of what you achieve here, you still have a problem (see #22192 for details) where you cannot use Go DLL from Go executable. How you are going to overcome that?

Alex

[cchamplin]

Thanks @alexbrainman

So my suspicion has been that what we are seeing is there is a runtime.* in the plugin dll and a runtime.* in the host executable. Which would explain why the memory for lastmoduledatap is not correct from the perspective of the plugin. I had forgot about #22192 but that ticket also pretty much confirms that suspicion.

It looks like what we need to replicate is the -rdynamic gcc flag. Which I don't believe gcc on windows supports.

So where do we go from here... What I'm trying to work on is having the host executable (though really we just need the base go internals) export all of its symbols (-Wl,--export-all-symbols) then using those via dlltool to build a library file that plugins link against (this is where I am now). Once we're linking against an external library for the "shared" go runtime I'm hoping we'll see some forward movement. I haven't gotten it to work quiet yet and I suspect it's because I need to stuff all of the go runtime symbols into .idata (windows import table) vs where it normally lives (.data,.text) for the plugin dll. Right now the whole thing is a lot of manual process and it may require that the plugin host be externally linked on windows for plugins to work (not the end of the world, I suspect that when building a host executable we can check for usage of the plugin symbols and switch to external linking automatically)

My bigger concern is whether or not we can link against a general go runtime in the plugin vs linking against the go runtime provided by the plugin host executable. I suspect we can, I don't really see any reason this shouldn't work, but I fear the plugin host may also have to be linked against the same general runtime...unsure though.

[cchamplin]

I see about the same. I could not stop gdb in the DLL loading code - I do not know how to do it. I tried breakpoint and rwatch. But I used objdump -d, and I can see this

What I'm doing is:
b go.link.addmoduledata then si from there to step forward until I know where the segfault occurs. If this is not working you may try b link.addmoduledata which is what the symbol is showing up for me as on *nix. I'm not sure why there are different.

[alexbrainman]

So my suspicion has been that what we are seeing is there is a runtime.* in the plugin dll and a runtime.* in the host executable. Which would explain why the memory for lastmoduledatap is not correct from the perspective of the plugin. I had forgot about #22192 but that ticket also pretty much confirms that suspicion.

I am not sure what you are saying here. As far as I understand runtime.* variables in executable and dll do not share the same addresses - they are independent of each other. The problem described in #22192 is different - both executable and dll use the same TLS register ( https://github.com/golang/go/issues/22192#issuecomment-335347282 ).

It looks like what we need to replicate is the -rdynamic gcc flag. Which I don't believe gcc on windows supports.

I am not familiar with gcc enough to comment. But from what I can gather, -rdynamic gcc flag provides support for dlopen. But all Windows DLLs can be loaded "dynamically" using LoadLibrary Windows API. So I am not sure we need -rdynamic gcc flag.

I haven't gotten it to work quiet yet and I suspect it's because I need to stuff all of the go runtime symbols into .idata (windows import table) vs where it normally lives (.data,.text) for the plugin dll.

.idata always contains very specific data - it is effectively relocations that Windows uses when it loads / initialises executable or DLL. .idata lists all DLL names / function names used by executable or DLL and some refs for Windows to write these function addresses during load.

I would not be surprised if existing Go DLLs already has all functions exported - so you could call them via LoadLibrary / GetProcAddress already. You just need to pass parameters correctly to them. And that is what I expect plugin mode achieves. But I could be wrong.

b go.link.addmoduledata then si from there to step forward until I know where the segfault occurs. If this is not working you may try b link.addmoduledata which is what the symbol is showing up for me as on *nix. I'm not sure why there are different.

This is what I do

c:\Users\Alex\dev\src\github.com\cchamplin\plugin_test>gdb plugin_test.exe
GNU gdb (GDB) 7.8
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-w64-mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from plugin_test.exe...done.
(gdb) b go.link.addmoduledata
Function "go.link.addmoduledata" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y

Breakpoint 1 (go.link.addmoduledata) pending.
(gdb) b link.addmoduledata
Function "link.addmoduledata" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y

Breakpoint 2 (link.addmoduledata) pending.
(gdb) r
Starting program: c:\Users\Alex\dev\src\github.com\cchamplin\plugin_test\plugin_test.exe
[New Thread 6200.0xe2c]
warning: Can not parse XML library list; XML support was disabled at compile time
[New Thread 6200.0x858]
[New Thread 6200.0x1f8c]
[New Thread 6200.0x25d8]
[New Thread 6200.0x420]
[New Thread 6200.0x2be4]
[New Thread 6200.0x24a0]
Starting
Module Path: c:\Users\Alex\dev\src\github.com\cchamplin\plugin_test\eng\eng.so

Program received signal SIGSEGV, Segmentation fault.
0x000000006fdf7f4c in ?? ()
(gdb) bt
#0  0x000000006fdf7f4c in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) disas
No function contains program counter for selected frame.
(gdb)

I do not see how I can use si command. Should I break on 'main.main' first, and then use si? That would take forever.

And, like @mattn said, you should replace your C code with single syscall.LoadLibrary call. For the purpose of your debugging, it does the same thing.

Alex

[cchamplin]

I am not sure what you are saying here. As far as I understand runtime.* variables in executable and dll do not share the same addresses - they are independent of each other. The problem described in #22192 is different - both executable and dll use the same TLS register ( #22192 (comment) ).

This is not what I am seeing on *nix. Here is a GDB run-through that unless I am mistaken shows that at a minimum runtime.lastmoduledata and runtime.firstmoduledata are at the same address in both contexts https://gist.github.com/cchamplin/abb9468e9b89e94d14cc0e2c4bafdaaa

IN main.main()

(gdb) x 0x0080a300
0x80a300 <runtime.lastmoduledatap>:     **0x0080f460**
(gdb) x *0x0080a300
0x80f460 <runtime.firstmoduledata>:     **0x00575fc0**

IN plugin->runtime.addmoduledata

(gdb) info registers
rax            **0x80f460** 8451168
rbx            0x7ffffa089800   140737388255232
rcx            0x836560 8611168
rdx            0x7ffffffee108   140737488281864
rsi            0x7ffffffee0f8   140737488281848
rdi            0x7ffffa16f200   140737389195776
rbp            0x1      0x1
rsp            0x7ffffffeda78   0x7ffffffeda78
r8             0x7fffff7e09d8   140737479838168
r9             0x7fffff7e09d8   140737479838168
r10            0x0      0
r11            0x0      0
r12            0x7ffffffee0f8   140737488281848
r13            0x7ffffffee108   140737488281864
r14            0x7ffffa089808   140737388255240
r15            **0x80a300** 8430336
rip            0x7ffff9e700cc   0x7ffff9e700cc <runtime.addmoduledata+12>
(gdb) x 0x0080a300
0x80a300 <runtime.lastmoduledatap>:     **0x0080f460**
(gdb) x *0x0080a300
0x80f460 <runtime.firstmoduledata>:     **0x00575fc0**

If you look at the values of R15 and RAX you see the addresses match.

I am not familiar with gcc enough to comment. But from what I can gather, -rdynamic gcc flag provides support for dlopen. But all Windows DLLs can be loaded "dynamically" using LoadLibrary Windows API. So I am not sure we need -rdynamic gcc flag.

My understanding of rdyanmic is that it is exporting symbols from the host in such a way that the plugin is able to use those symbols. Again I could be mistaken but that's my understanding

    Pass the flag -export-dynamic to the ELF linker, on targets that support it. This instructs the
    linker to add all symbols, not only used ones, to the dynamic symbol table. This option is needed for
    some uses of "dlopen" or to allow obtaining backtraces from within a program.

.idata always contains very specific data - it is effectively relocations that Windows uses when it loads / initialises executable or DLL. .idata lists all DLL names / function names used by executable or DLL and some refs for Windows to write these function addresses during load.

So going off the assumption of the plugin needing to access loader/host symbols I believe idata would be the appropriate place for those symbols (inside the plugin). This is what I see when creating a C/C++ plugin

# common.h:
extern int testThing;

# loader.cpp
// gcc loader.cpp -o loader.exe 
// dlltool --export-all-symbols loader.exe -z loader.def
// dlltool -d loader.def -l libloader.a -D loader.exe
#include  "common.h"
 __declspec(dllexport) int testThing;
...
hinstLib = LoadLibrary(TEXT("plugin.dll")); 
...

# plugin.c
// gcc -c -o plugin.o plugin.c
// gcc -o plugin.dll -s -shared plugin.o -Wl,--subsystem,windows -LI:\development\\dynamicdll_gcc\ -lloader
#include  "common.h"

loader.exe EXPORTS (.edata)
testThing

plugin.dll IMPORTS (.idata)
testThing

I do not see how I can use si command. Should I break on 'main.main' first, and then use si? That would take forever.

I'm not sure why your experience is different than mine. I've provided a gist the debugging on windows if that helps here: https://gist.github.com/cchamplin/2f930b60551aece7de4118f168390093

[alexbrainman]

This is not what I am seeing on *nix. Here is a GDB run-through ...

I am, probably, wrong, but imagine you have runtime.addmoduledata function in your Go program.

When you compile Go executable, the function code endup somewhere in .text section of PE executable.

When executable runs, Windows loads executable file starting at some fixed address (that is hardcoded in the Go linker). So .text section will always starts at the same address (.text section is first section), and runtime.addmoduledata will starts somewhere after that address.

Similarly when Go DLL is built, Go linker puts runtime.addmoduledata somewhere in .text section of an object file and passes it to GCC. GCC copies runtime.addmoduledata code into .text section of DLL file. When .DLL file is loaded by any executable, Windows loads file at some random address - it has to be random, otherwise different DLLs will conflict with each other. So, similarly to executable, .DLL file .text section will endup at fixed offset after that random .DLL address. And runtime.addmoduledata will apears some offset after that.

So if Go executable loads Go DLL, we will have 2 copies of runtime.addmoduledata loaded at different addresses.

But, I don't see how this is relevant to our conversation.

If you look at the values of R15 and RAX you see the addresses match.

I think the difference you see between Linux and Windows, is that R15 in runtime·addmoduledata is set to 0. See this comment about R15

https://github.com/golang/go/blob/ba2e8a629b36e43cc27b23470b631a1dfee0900f/src/runtime/asm_amd64.s#L1342-L1349

I do not know how R15 is used to access global variables. And who sets R15. Maybe others will help.

I'm not sure why your experience is different than mine. I've provided a gist the debugging on windows if that helps here: https://gist.github.com/cchamplin/2f930b60551aece7de4118f168390093

It does help. I downloaded later gdb version, and it works similarly to yours. Thank you very much.

Alex

[cchamplin]

@alexbrainman

So if Go executable loads Go DLL, we will have 2 copies of runtime.addmoduledata loaded at different addresses.

But, I don't see how this is relevant to our conversation.

It's not so much runtime.addmoduledata but runtime.lastmoduledatap and runtime.firstmoduledata. When we have two versions of these it's causing it to blow up, since in the plugin lastmoduledatap isn't initialized (this is handled by the plugin host)

Anyways I've made real progress and think that the path I'm on is workable here's where we're at, if anyone notices anything that sounds wrong please let me know:

• These comments speak for Windows - Linux may be different in many respects
• Technically the host and the plugin don't both need copies of the go runtime in their binary
• The plugin ideally should not include the whole Go runtime but instead should utilize the runtime included in the host
• The PE format is different in that there isn't really a GOT which appears to be how plugins on *nix are accessing host memory
• If we can make the runtime.* symbols from the host available to the plugin during linking and execution most of the existing plugin code should just work

Current technical steps:

1. build the host

go build -ldflags=all="-linkmode=external -extldflags=-Wl,--export-all-symbols" greeter.go

1. Create symbol definition (this might be temporary, there may be ways to get gcc to produce the shared library)

dlltool --export-all-symbols greeter.exe -z runtime.defs

1. Cleanup runtime.defs (Basically remove non runtime.* symbols)
2. Create library

dlltool -d runtime.defs -l libruntime.a -D greeter.exe

1. Adjust golang linker to produce object files that gcc will correctly link up for buildmode=plugin
   a. Mark every symbol as extern
   b. Set the value for the runtime.* symbols to 0x0 instead of their normal locations
   c. Prefix runtime.* symbols with __imp_ (gcc uses this to correctly build the idata/import table)
   d. Set the section for all runtime.* symbols to idx 0 (should be .text I think)
2. Build the plugin

go build -ldflags=all="\"-extldflags=-LI:\\path\\to\\dir\\ -lruntime\"" -buildmode=plugin .\eng

Where we are now:

So things build and if you run the host (greeter.exe) we are able to get past runtime.addmoduledata. It doesn't directly segfault but we panic when trying to read the pkghash data. Haven't had time to investigate yet.

[rocket049]

I found sombody have a new idea:
"github.com/dearplain/goloader"
it seems better.

[blaubaer]

Ok, what we can do to move that issue forward?

[bradfitz]

@blaubaer, it depends on whether you know how to fix it I guess.

If so, see https://golang.org/doc/contribute.html
If not, see http://golang.org/wiki/NoPlusOne

Really, this is waiting on somebody who knows the subject area and wants to send some code.

[marcogrecopriolo]

from https://github.com/marcogrecopriolo/sqsl/blob/master/oslib/coslc.c

/*
** dlopen
*/
DLLDECL void *fgw_dlopen(char *f)
{
return (void *) LoadLibrary(f);
}

/*
** dlsym
*/
DLLDECL void *fgw_dlsym(void *h, char *s)
{
return (void *) GetProcAddress(h, s);
}

/*
** dlclose
*/
DLLDECL void *fgw_dlclose(void *h)
{
FreeLibrary(h);
}

static char errbuf[20];
/*
** dlerror
*/
DLLDECL char *fgw_dlerror()
{
int rc;

rc=GetLastError();
sprintf((char *) &errbuf, "%d", rc);
return (char *) &errbuf;

}

This is really all there is to it. Could we please have it now?
Also, could we have func(p *Plugin) Close()?

These two would make golang user defined functions inside couchbase's N1QL very easy to implement.

[xtxy]

any plan for this feature?

[goapifirst]

Seems as Go is one of the hot languages and growing fast now, it would benefit to support plugins on all three platforms and hopefully in a way that doesn't require the main app to be compiled using the same version of go. That is a lesser evil, but would be nice if say an app compiled in go 1.14 and load a plugin compiled in 1.17 and not crash, and vice versa. Here is to hoping someone will add this feature for Windows and see some examples of building a simple app that uses a plugin that is also built on all 3 platforms.

[r3inbowari]

Maybe someone who needs to promote this plan...

[davecheney]

Thank you to all the commenters. Can I remind everyone of the No +1 policy. It's clear that there is a desire for pluggin support for windows, there is no doubt about that, but at the moment nobody from the Go team or the community has stepped up to do the work to implement this feature.

[goapifirst]

@davecheney Understood.. however I am a little surprised that this is something the Go team put in a release.. half baked. I would have thought by now, multiple releases later that it would have been on a list somewhere to get completed and not left hanging given it was implemented already for one platform then fixed for a 2nd. I mean if it was a 3rd party addon/lib/etc and built this way, sure.. but that it is part of the official golang distribution.. but not complete.. seems like it should bubble up to the top on things to finish. I dont know if there are other features like this in the language/core that are half baked and only work on one or two platforms, but Golang is largely understood to produce binaries on all three platforms, so to leave out support for one on a feature that, at least to me is actually quite a big deal.. but is receiving less use because it's not complete, just seems out of ordinary for an otherwise pristine piece of kit. Again I'll say.. if this was not part of the core language and already in a release for a while now, I think many could understand. I have to believe someone among the many of you amazing developers could spend a little time on this and figure it out.

[ianlancetaylor]

It is really indisputable that plugins are a half baked feature. They have significant problems even on platforms where they (mostly) work. It was perhaps a mistake to add them at all. I think I approved the change to add plugins myself; I'm sorry for the trouble they are causing.

[goapifirst]

@ianlancetaylor I am going to assume it would be difficult at best to remove the feature at this point? Maybe Golang 2.0 can remove that feature.. or do it right. I am a nut for "plugins" as they do have a lot of value.. being able to load code dynamically as needed.. and more so in Golang if it worked the same across all platforms, would be amazing. There is clearly ways for this to work as languages like C can load .dll and .so dynamically on all platforms. I am going to assume then that it is more a matter of not having the right expertise within the team or anyone for that matter to not only get it working, but in a way that it is consistent across all platforms.
I will say though that today, with things mostly done via PWA/web based apps, I can see why it may not have much significance in Go where it is not typically used for desktop based apps where the notion of plugins is more common (e.g. audio plugins, editing plugins in video editors, game addons, etc). The RPC style plugin engine seems usable in most situations that don't need extreme performance like real time audio processing, etc.
Still.. if it can't be removed.. would be great to see it resolved. :)

[ianlancetaylor]

Unfortunately it's more subtle than just generating a DLL. Given the way that Go works, the plugin and the main executable will inevitably both import some of the same packages. Those packages will then exist twice as code, but in the general case they must share the same data variables. This is straightforward in ELF, where a symbol in the main executable will override and replace a symbol in a shared library. This is not straightforward in the PE/COFF format used on Windows. I expect that there is some way to do it, but there is no obvious simple way.

[marcogrecopriolo]

Here's my stab at it: https://github.com/marcogrecopriolo/plugin
Little snag - windows doesn't support buildmode plugin or shared.
I've tried testing using a c-shared built package, but the the next snag I hit is that windows doesn't seem to keep track of dynamic module data, so we panic in lastmoduleinit().
I guess somebody will have to enable plugin builds for windows?

[g13013]

@ianlancetaylor for what it worth, Go plugins are useful for us!