Go: cmd/go: drop support for binary-only packages

[rsc]

Binary-only packages are increasingly hard to support safely. There is no guarantee that the compilation of the binary-only package used the same versions of the dependencies that the final link does (and it would probably be too onerous to insist on that). As a result, the binary-only package may have been compiled using escape analysis results or inline function bodies for dependencies that are no longer accurate. The result can be silent memory corruption.

My memory is that we added binary-only packages originally so that a professor could hand out binary solution sets so that a student who didn't finish lab 2 could still move on to lab 3. I don't know why else anyone uses them anymore, but they're probably going to break more and more as the compiler gets more sophisticated.

Should we just remove them? If not, why not? What do people need them for?

See also #28146 (less severe).

/cc @bcmills

[bradfitz]

I'd like to see them removed, mostly for code simplicity.

For the professor use case, perhaps source obfuscation of some form would be sufficient.

[rsc]

Looking for people who use binary-only-package and might be able to explain what they use it for.

@ianlancetaylor filed #23473 (but presumably passing along another report).
@triztian filed #26590.
@hilyjiang filed #24318.
@motomux filed #26875.
@joshlf filed #16841.
@jerrybean filed #21451.

/cc @dneil @dsnet for knowledge of any potential Google-internal usage.

[neild]

I'm not aware of anyone using binary-only packages inside Google. Doesn't mean someone isn't, of course.

[cznic]

I'm in favor of the proposal, nonetheless I guess the important property of binary packages is they make some closed source package vendors feel "safe" about their IP.

[booss]

I'm in favor of the proposal, nonetheless I guess the important property of binary packages is they make some closed source package vendors feel "safe" about their IP.

That or plugins need to not panic (when loading) and complain about different versions of the same package and also offer unloading support (i.e: find a way to sever the link with the runtime and free evreything).

Vendoring binary packages is useful if you're selling an SDK or toolkit.

[triztian]

Hi there, thank you for considering us!,

In our case we use it as part of our offering to Enterprise customers. Being able to provide non-source distributions and as a way to hide some of our I.P clears us with the IT departments of such companies.

Our biggest use is being able to provide a static library (the BOP package) to third parties without providing our source, we provide an SDK that _should_ work without any connection to our Servers, hence we package the SDK as binary-only-package that includes some proprietary I.P. that is licensed.

While that approach is not the most secure way to protect our source, it is the right balance right now, we just don't include more sensitive functionality in what we distribute.

Another important characteristic is that being able to have a higher level of friction of preventing modification to what we distribute; i.e. even if we provide non-IP that has to function in a specific way, not having BOP's would make easier for third parties to modify the SDK.

I'm not entirely in favor of dropping support for binary-only-packages because I believe that many other languages support a way of distributing packages that don't have the original source in them:

• Java/Android - Jar/aar files
• Swift - Frameworks
• C/C++ - static libs + headers

[ianlancetaylor]

23473 was for an internal Google use case (internal reference number 72160357) that was resolved by changing to not use binary packages.

[rsc]

Posted to golang-nuts to cast a wider net for use cases.
https://groups.google.com/d/msg/golang-nuts/juPzaRDVB9c/6PYP56-0CAAJ

[wsc1]

In general I'm in favour of giving incentive to provide source. But there are legitimate binary-only cases related to not only IP but security.

What relationship would removing support for BOPs this have to plugins and related build modes (shared/c-shared/c-archive)? Could one still provide a binary in any of these forms and be able to use that? If so, then a simple solution for at least some BOP use cases would be to use a wrapper package that loads a plugin, or uses -linkshared or a c-archive via cgo and the plugin or c-archive would be outside of go get/modules and distributed separately as binary.

I have no plans to distribute such things, but would like to be able offer the capacity to others in some form. If something like the above workarounds worked, then that would provide a way to drop BOP
support and say "use workaround ..." for those who want it.

At the same time I understand there are still some issues with these other binary distribution mechanisms and modules.

[rsc]

Thanks for asking about plugins. There are no plans to remove plugin support, and suggesting the use of plugins as binary blobs instead of binary-only packages makes sense to me. Plugins have many of the same concerns, but they already have a proper check for mismatched API (avoiding the silent memory corruption problem) and it's always better to have one mechanism than two.

[joshlf]

Are there any performance concerns for plugins? E.g., I would imagine that you can certain link-time optimizations with binary-only packages that aren't available for dynamically loaded code?

[booss]

I agree that binary-only-packages are problematic and should be removed.
I tried to use plugins but you can't unload them.
I decided to live with that and restart the process in case the plugin had to be reloaded but then:
You can't load a plugin if you vendor your dependencies and one of this dependency is shared with the host.
Right now I'm using go tools (compile and link) and reuse the .a but that's kinda ugly

[ianlancetaylor]

@joshlf It's hard for me to imagine any real case in which plugins are noticeably less efficient than binary-only packages. It's possible to construct an artificial case, but I doubt that would match any real implementation.

[wsc1]

Thanks for asking about plugins. There are no plans to remove plugin support, and suggesting the use of plugins as binary blobs instead of binary-only packages makes sense to me. Plugins have many of the same concerns, but they already have a proper check for mismatched API (avoiding the silent memory corruption problem) and it's always better to have one mechanism than two.

Great to hear. It seems to me it wouldn't be too much to adapt the source stubs from a BOP to wrap a plugin and an init() to load it from somewhere, and well worth it to both BOP users and Go maintenance.

[rsc]

Will leave this open for another week but it sounds like we can retire binary-only packages in favor of using plugins. We could advertise in the Go 1.12 release notes that it will be the last release to support binary-only packages and they will be removed in Go 1.13.

[andybons]

Accepted since we waited a week and nothing came up/no objections were raised.

Per discussion with @golang/proposal-review

[edburns]

As a corporate programmer, the closed-source use case is very important. A language that is so RMS idealism friendly that it doesn't even allow one to use closed-source dependencies is a very hard sell in many corporate environments.

[mvdan]

@edburns see the discussion above; using compiled binaries will still be possible via plugins. binary-only packages are harder to develop, maintain, and use, as can be seen above.

The later stages of this thread were about finding any reason why plugins aren't a good replacement for binary-only packages, to which there wasn't a clear answer.

[ianlancetaylor]

@edburns Thanks for the comment. Do you have direct experience of this? How do those corporate environments handle languages like Python or Javascript?

[edburns]

Consider the case when a corporation is using a licensed closed-source software package as a dependency on an internal project. Right now, if the vendor of a closed-source package wanted to make their licensed package useful to go programmers, their only option is to author it as a plugin. As stated elsewhere, this has drawbacks: complexity of .so linking, lack of non Linux support, etc.

[mvdan]

make their licensed package useful to go programmers

I'd argue that using a binary-only package was never really an option. The binary would require an exact version of Go, as well as an exact version of all the packages it depends on. It's certainly possible, but I wouldn't say it's ever been a good or easy way to distribute software. The intended purpose, the professor scenario, is a much more restricted use case.

complexity of .so linking

Please expand on this. If you believe there's a bug or something to improve in the plugin package or the go tool, you should open a separate issue.

lack of non Linux support

What particular platform is noticeably missing plugin support? Assuming you mean Windows, that's being tracked in #19282. I believe effort would be better invested in issues like that one, instead of on maintaining binary-only package support.

[ianlancetaylor]

@edburns Thanks. Personally I am more interested in direct experience than in hypotheticals. We can construct a number of hypothetical situations, but in the end they won't tell us what to do.

[oszika]

It is possible to use vendoring with plugins (without set package import path)? The use of multiple plugins is hard because they need to use exactly the same version of common dependencies. Would it be possible to use the latest minor version for each dependency otherwise?
Edit: the last point seems to be impossible to do because we cannot unload a plugin.

[twyvip]

1. I have one file in /root/work/go/src/say named say.go
   package say
   import "fmt"
   func Say(s string) string {
   return fmt.Sprintf("hello %s", s)
   }

2. I try to build it to .a file go build -i -o $GOPATH/pkg/linux_amd64/say.a

3. change /root/work/go/src/say/say.go to

//go:binary-only-package

package say

1. another file /root/work/go/src/say/work/v1.go
   package v1
   import (
   "fmt"
   "say"
   )
   func SayToBob() {
   fmt.Println(say.Say("Bob"))
   }

2. the main file /root/work/go/src/say/work/main.go
   package main
   import (
   "say/work/v1"
   )
   func main() {
   v1.SayToBob()
   }

3. run main.go go run /root/work/go/src/say/work/main.go
   go run main.go
   hello Bob
   that's right

7.but in fact, i need to build SayToBob as plugin to hot reload,run
go build -buildmode=plugin -o /root/work/go/src/say/work/v1/plugin.so /root/work/go/src/say/work/v1/v1.go
missing or invalid package binary for binary-only package say

[bcmills]

@twyvip, how does your comment relate to this proposal?

[massimiliano-mantione]

FWIW, I have a "convenience" use case for binary-only packages: cross compiled cgo packages that are inconvenient to compile on the host because everybody would have to install the relevant C cross compiler (for the target).

Concrete case: I am starting to write a go wrapper for the librobotcontrol C library for the beaglebone blue.

That C library makes no sense on the host (be it windows, mac or linux).
AFAIK to cross compile the wrapper I'd need to copy the relevant header files and libraries on the host, and also provide cgo a C cross compiler.

I was strongly considering building the wrapper natively on the beaglebone (very easy to do), and just redistribute the compiled object file so that users do not need to set up the cgo cross compilation environment.

In this case "users" would be a group of 15 years old highschool students I am mentoring for a school-related robotics project.

Anyway I can probably do this cgo wrapper as a plugin, with a bit of boilerplate code to export a type safe interface to the wrapped functions.

Do you still think to deprecate binary-only packages?
If yes I'll go the plugin way, since that would be the only supported option in the future.

[bcmills]

@massimiliano-mantione, a plugin sounds like the right approach for your use-case.

[gopherbot]

Change https://golang.org/cl/152918 mentions this issue: doc: announce the end of support for binary-only packages

[ianlancetaylor]

@massimiliano-mantione Another approach you can use is .syso files. Compile all the C code into a single object file per support platform, and name it something like c_GOOS_GOARCH.syso. Then the Go code can be the same as always, and can use cgo to call into the C code provided as a .syso file.

[gopherbot]

Change https://golang.org/cl/159557 mentions this issue: cmd/go: mention that binary packages are going away

[dzpt]

@rsc Using plugin is total different thing with binary-only packages.

The good case is i want to protect web framework source code like beego (as an example) there is no way to archive the same thing with plugin, it's painful.

Imagine your code since:

import (
    "framework"
    "framework/config"
    "framework/server/middleware"
    "framework/util"
)

framework.Abc()
util.Xyz()
util.Sort()
config.Read()

turns to this for using only ONE method

import (
    "fmt"
    "plugin"
    "path/filepath"
)

all_plugins, err := filepath.Glob("frameworks/*.so")
if err != nil {
    panic(err)
}

for _, filename := range (all_plugins) {
    p, err := plugin.Open(filename)
    if err != nil {
        panic(err)
    }

    symbol, err := p.Lookup("Sort")
    if err != nil {
        panic(err)
    }

    sortFunc, ok = symbol.(func([]int) *[]int)
    if !ok {
        panic("Plugin has no 'Sort([]int) []int' function")
    }

    sorted := sortFunc(numbers)
    fmt.Println(filename, sorted)
}

We have to check for a thousand methods that one framework provided.

If you have ever used C++ or Swift it works like binary-only package. Using plugin in Go is painful, code will be 100 times longer while methods aren't exposed to be used. It's suitable for CLI app rather than large scale framework.

Its all about IP, i want to sell the SDK or want other developers to use my framework without copying sourcecode to use on another project without paying.

[gen0cide]

@aaveidt I'd argue if protecting your IP is that important to you, why wouldn't you maintain a proxy package to include the boilerplate required to load your plugin on behalf of your customers?

If the IP is locked up inside the precompiled plugin, it doesn't seem like providing your customers easy to use interfaces should be that much trouble.

[dzpt]

@gen0cide i'm not argue about plugin isn't capable to protect my IP.
I'm telling you that using plugin and binary package are like comparing an axe and a knife.
Both can be used for chopping tree but would you use a knife for it?

It takes so much time to convert one framework to a plugin.
And we have to check for each symbol in each plugin, redeclare interfaces and types ... Plugin is suitable for using one/few package(s) or CLI app not a whole framework.

Plugins have many of the same concerns, but they already have a proper check for mismatched API (avoiding the silent memory corruption problem) and it's always better to have one mechanism than two.
@rsc taking away an axe and left the knife is the drawback, you should remake or sharpen an axe.

[fbnz156]

This change will disrupt my distribution pipeline beyond belief.

Distributing binary packages is crucial to any commercial SDK-style use case and insisting on using plugins is going to force a JNI-like wrapper, abstracting structs and more.

Plugins are, by their very nature, supposed to be entirely self contained with few exports. SDKs are by nature not and are supposed to heavily integrate with application logic throughout. Forcing these to effectively use reflection and a generated proxy layer is absurd.

Why not just integrate binary-only builds with go modules? Just include the go.mod file in the zip and assert that the versions are the same? It can't be more than a 20 minute job.

Failing that, why not distribute all the specific version dependencies as binaries? And allow multiple packages to exist within one compiled zip build.

There's so many possible solutions. Jumping to "get rid of it and don't replace it" is absurd, and is the kind of executive decision that will lead to diverging forks.

Suggesting that this feature only has uses in education is like suggesting that JAR files are only used by people using Guava. It's silly.

Nobody has objected yet because most people don't pay attention to the issue tracker. The first I'm hearing about this is in 1.12 changelog.

[ianlancetaylor]

@fabian-f This was discussed on the golang-nuts mailing list, as mentioned above. We tried to raise awareness of this issue.

Note that binary packages continue to work. What will stop working in the 1.13 release is binary-only packages. That is, you can distribute a package binary, but you also need to distribute source that tools can use to rebuild when appropriate. Can you describe your requirement for binary-only packages? Because right now they don't really work.

[fbnz156]

@ianlancetaylor I distribute an SDK which runs as a server (started by the clients main package) and allows clients to hook events and interact with exported methods and struct fields.

Binary-only packages are used so that my licensing model can work since the SDK itself runs independently, and distributing source would allow clients to continue using the SDK indefinitely with slight modification.

I haven't noticed any issues functionality wise.

[mvdan]

@fabian-f please show an example of an application which can use binary-only packages, but not plugins. This thread is getting more and more critical replies, but none have explicitly said why they can't use plugins, as best as I can tell.

Yes, plugins require a bit more boilerplate, but that's also why they're much easier to support and maintain in the toolchain. Like previous comments in the thread explain, binary-only packages are vastly more complex, so it makes little sense to keep supporting them when we have plugins.

[fbnz156]

@mvdan I will put the concept I described into code a bit later.

The thing isn't that plugins "can't" be used, it's just that it's a performance hit. To get full functionality you need to use reflection far too much and it turns trivial code into really complex code. (Create a struct and set a field for example).

Maybe the plugin system could, one day, be a swap in replacement - if they're changed to be "real go" instead of reflection-like access.

[dzpt]

@mvdan the examples are above, don't know what you ask for.
Using plugin isn't only a bit more boilerplate, code shown above too.
Its 100 times longer

[fbnz156]

@mvdan

Consider the following:

//go:binary-only-package
package test_binary

import "log"

type HelloRequest struct {
    Name string
}

func SayHello(req HelloRequest) {
    log.Println("Hello,", req.Name)
}

Accessing this via binary-only packages is seamless
test_binary.SayHello(test_binary.HelloRequest{Name: "Fabian"})

By contrast, I don't even know if this can be done in plugins without first creating a factory method for HelloRequest?

I can't test it since I'm on Windows, which plugins don't support.

Binary packages are a requirement of programming languages in general, and those languages that lack it (Python) do suffer for it. Removing them is going to kill commercial SDKs and middleware.

[cpipero]

I don't know if this matter anymore at all, but we planned an entire year of development for a commercial SDK which was entirely protected by the binary only package. We got two customers using the beta (and paying for using it) and now we are basically forced to abandon a project which cost us in excess of 120K so far (cost, not revenue opportunity).
The plugin option is not even remotely attainable (as other as observed).

I wonder how those features get deprecated, that's why this year we are sticking with the standards (C, C++) which are backed by stronger governance.

[ianlancetaylor]

Thanks for the input. We'll consider it.

The main reason for deprecating this feature is as outlined above: it does not work reliably.

[yslim]

Is it true that Go can't distribute lib without source code? If then, please write it on first page of golang.org not to start golang. I think it's non-sense not supporting binary lib distribution.

[fbnz156]

I'm in a similar, albeit of lower magnitude, situation to @cpipero.

@ianlancetaylor I totally get the need to burn a bad code base to the ground, but there needs to be a suitable drop-in (or close to) replacement that A) has the same functionality and B) works in a similar way.

Whilst plugins provide, on an abstract level, similar functionality, they are lacking completely when it comes to type definitions. It's just not there at all. Then there's the objectively horrible syntax which is just begging to be generated (which is a possible but not ideal workaround).

[triztian]

@ianlancetaylor would the "plugin" package eventually be able to at least support loading structs declared in the plugin?.

I think that would further make plugins a replacement for binary-only-packages.

[cpipero]

We looked at plugin and it's not so practical for our use.
You need to call plugin.Lookup("FooType") and then cast the result to FooType to be able to construct a type. We've got a lot of types (we are selling an SDK).
Again, it's all in the economics: this issue is costing us a lot...

[yslim]

Today, I studied plugin.
I think plugin concept is very similar to _swig_.
All sharing types should be treated as interface.

func main() {
shared.LoadGreetingPlugin("plugin/hello.so")

greeter1 := shared.NewGreeting("test1")
greeter2 := shared.NewGreeting("test2")
greeter2.SetName("test3")

greeter1.SayHello()
greeter2.SayHello()
}

https://github.com/yslim/go_plugin_test

In my case, plugin will be a good option for replacing binary-only-package.

[dzpt]

In my case, plugin will be a good option for replacing binary-only-package.

@yslim Have you used binary-only package? Its 100 times shorter.
https://github.com/yslim/go_plugin_test/blob/master/shared/greeting.go

Your main.go and shared/greating.go files have 45-50 lines of code to call only 1 SayHello() method.
Using binary-only package only costs 4

package main
import hello
greeter := hello.NewGreeting("test1")
greeter.SayHello()

Then i can call any exported methods effortlessly without writing any line of code: greeter.A() , greeter.B() ...
What would you do with plugin, you have to redeclare each of them in your Greeting interface.

One framework or big package can have hundred types, and exported methods.
Try to port any package like gin, chi, gorm, beego to plugins to see the difference.
Using plugin isn't practical at all.

[gen0cide]

I completely disagree with your position @aaveidt and @fabian-f for a number of reasons.

You're continuing to assert that binary-only packages are the only option because they allow your users easy access to code that you write, but that you will not share with them. Where I struggle to sympathize is in that you're doing this exclusively for Go packages. This isn't for c-shared libraries that have symbols exported and documented. You're asking Go developers to implement your binaries under the following conditions:

1. No access to the underlying source code. Hopefully you generated some kind of a Godoc, but even then, I'd find myself frustrated as Go developer giving you money to use your code. Not a day goes by that I don't click through an Anchor into code in Godoc. And I think that's a paradigm that has made Go successful - it has the best documentation story of any language.

2. Very poor compatibility. As has been noted by many issues here, I'd be extremely upset as a developer if my vendor continues to provide me libraries that suffer from inconsistent, buggy behavior.

What I'm getting at is that while it is 100% your choice to decide that theoretical intellectual property protection is more important than your developers experience, I do not understand how somehow removing BOPs when Go has provided you with a much more stable Plugin system[1].

Your biggest complaints about the Plugin system relate to "effort". It definitely will require some work to evolve your codebases to comply with the new supported standards, but repeatedly arguments have been made about how this would be an undue burden on your users. Ultimately, your users, who are Go developers, are burdened by your choices, not the Go ecosystem's. If you valued their experience above all else, writing some go/ast, go/parser and go/printer code to programmatically generate intermediate plugin interfaces that you could share with users doesn't seem like too much to ask.

Especially when the result will be more stability, better documentation and usability, and easier forward compatibility. I support @ianlancetaylor and @rsc on the job done. As much as they value API stability and compatibility, this is the type of deadweight that drives poor developer experiences in the ecosystem. They recognized it and it's time for it to deprecate into the past.

[1] I realize that right now that Go doesn't have Plugin support for Windows. But so far, a significant amount of push back has been from people talking about "web frameworks" which overwhelmingly are developed and deployed on Linux.

[cpipero]

@gen0cide thank you for the valid points, which speak to ultimate stability and developer well-being.
Our problem is more of a practical nature, where a known feature was the base for us to lay out a legal and commercial framework for the distribution of a product.

That framework ceased to exist once no equivalent feature could be provided by us to our customer.

Again, I did not pay for golang development so I am not going to blame the ultimate decision. I just voiced the reason why I would vote for keeping a feature that works for us without any issue, understanding the risks but not having a whole lot of choices given the significant investment.

You say that writing a go/ast, go/parser, etc. is no big ask. In our case, we would probably refactor our entire API into a language that 1) supports the legal agreements in place (for us and our users), and 2) eliminates the risk of similar decisions by enforcing a more formal ISO/IEC process to make changes.

And in doing that (or if I were to follow your advice) it means to come up with a lot of money (of course, that subjective but for me anything more than 20K is a big ask).

Therefore, the point here is how to deal with existing SLA's.

With all that said, all I can do was to relate a real world scenario.

Thanks again for the advice.

[fbnz156]

@gen0cide Writing some AST magic to generate a proxy layer is no big ask. I've already started it in anticipation of this change. The problem is plugins aren't ready to act as a replacement yet. They're only supported on Linux and Mac (3 of my larger clients have to deploy to Windows), and don't support struct type definitions. I don't get why a handful of people are so eager to deprecate this feature without a single full replacement.

[ianlancetaylor]

@fabian-f I want to express again that, as we've said, the reason to deprecate binary-only packages is that they are not reliable. If accidentally misused, they can lead to silent memory corruption during execution. That is both unfortunate and difficult to fix.

[ianlancetaylor]

Let me add that I do understand that you have a use case for binary packages. Months ago @rsc asked for people to give examples of use cases that were not supported by plugins. Only in the last few days have they come forward. I understand that not everybody can track the Go mailing lists. Now that a couple of use cases have come forward, we will re-evaluate the decision and see if we can find a way to move forward. If anybody wants to address the technical issues that make binary packages difficult to support, that would be very helpful.

[cpipero]

Again,

I’m very grateful for the consideration

[rsc]

@cpipero and @aaveidt, from a "code hiding" point of view, the compiled .a file has a lot of source information embedded in it. A LOT. It might be equally practically effective just to run a code obfuscator on your code and distribute source. Perhaps much better, since it would hide more of the internal names.

If such a code obfuscator tool existed, would that work as a replacement for binary-only packages for you?

[mirtchovski]

If such a code obfuscator tool existed, would that work as a replacement for binary-only packages for you?

I don't know about the current cases, but the malware community would be very happy with such a tool to make reverse engineers' tasks slightly more difficult. cf. https://blog.malwarebytes.com/threat-analysis/2019/02/new-golang-brute-forcer-discovered-amid-rise-e-commerce-attacks/

[cpipero]

We ran that option to the customers (two) and both legal teams came back with “no”.

On Feb 27, 2019, at 2:53 PM, mirtchovski <[email protected][e-mail]> wrote:

If such a code obfuscator tool existed, would that work as a replacement for binary-only packages for you?

I don't know about the current cases, but the malware community would be very happy with such a tool to make reverse engineers' tasks slightly more difficult. cf. https://blog.malwarebytes.com/threat-analysis/2019/02/new-golang-brute-forcer-discovered-amid-rise-e-commerce-attacks/

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/golang/go/issues/28152#issuecomment-468005459, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABwUtxNqITLOdHZt5-6U23TDQ_as8-jcks5vRuI_gaJpZM4XXyuW.

[bradfitz]

@cpipero, you might need to help explain things to your legal team, then. You can't expect them to know how all (or any) compilers work.

[gen0cide]

@rsc, I'm supportive of exploration of Go obfuscation in general, but curious if it isn't trending a more broadly applicable solution near @fabian-f and @cpipero's comments regarding auto-generation of plugin scaffolding.

I spent years reverse engineering obfuscators for MMORPGs who used a wide variety of Java obfuscators, open source and commercial alike. Bytecode is easily represented in source - so in effect, even the best Java obfuscator is basically what you're proposing - a source obfuscator. Reversing these obfuscation techniques were common practice by myself and other teenagers. Also, at the end of the day, much of the obfuscation techniques that work well will diminish the optimization strategies of the compiler, as well as negatively effect performance of the compiled binary. This is now how most enterprise mobile obfuscators work today.

Nowadays, I work in information security and actively research and develop techniques of binary obfuscation using Go. I've spent a lot o time leveraging the power of go/ast and go/parser to this end. I love the theory of a richly featured obfuscation utility, but having gone down that road, I would anticipate supporting such a utility would be more cost than the benefits in this particular issue. Basically, this boils down to the fact that to really check the boxes for anti-introspection, anti-tampering, and preventing reverse engineering (especially in a situation where you're distributing obfuscated source), you'd need to write rules governing the re-implementation of every ast.Node in the tree. Note that to be effective, you'd need a wide library of these rules to look at not just individual nodes, but parent nodes and reference calling. Then, the obfuscator must randomly (or keyed with an secret IV) select strategies at every level, wrapping, rinsing, and repeating. This "code mixing" is all for not if the reverse engineer can simply document your re-write rules to perform the inverse on identified nodes.

I still believe the stability of the language ecosystem and compiler requires BOPs to be deprecated, but I do think there is room to innovate something on behalf of those who have use cases. How do you feel @rsc and @ianlancetaylor about the concepts of some sort of plugin scaffolding generator? I don't know where this lives, but I see the feature working something like this:

1. DeveloperA develops a plugin using currently recommended practices.
2. DeveloperA marks through a comment //plugin:export the types and functions they wish to expose.
3. DeveloperA runs builds the plugin but with an additional -export flag. The compiler produces 2 files - plugin.so (the compiled plugin), and plugin_export.go.

What happened? The compiler walked the AST looking for plugin exports, then generated a scaffolded export that implements those exported types and function calls using a standardized entrypoint. This go code, along with the plugin, could then be distributed to developers.

Thoughts?

[edits for grammer/spelling]

[cpipero]

Not my legal team: the customer’s.

You base your project on a set of documented features that keep all their IP concerns at bay. Then you go to them one year later saying “we can do two out three but we can’t keep the code in binary form as previously stated “.

Believe me, at those companies the last thing you want to do is to school their lawyers on computer science.

But we are getting off track. I don’t think people who are pushing back are questioning the technical reason for the decision. More than anything we are hoping for a more structured approach to change. And I have seen a few good ideas in the last hour or so.

Success is often a compromise

On Feb 27, 2019, at 3:15 PM, Brad Fitzpatrick <[email protected][e-mail]> wrote:

@cpiperohttps://github.com/cpipero, you might need to help explain things to your legal team, then. You can't expect them to know how all (or any) compilers work.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/golang/go/issues/28152#issuecomment-468013039, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABwUt1djKESR4JKOt4xDAAEEiXhGlhKzks5vRudogaJpZM4XXyuW.

[gen0cide]

If such a code obfuscator tool existed, would that work as a replacement for binary-only packages for you?

I don't know about the current cases, but the malware community would be very happy with such a tool to make reverse engineers' tasks slightly more difficult. cf. https://blog.malwarebytes.com/threat-analysis/2019/02/new-golang-brute-forcer-discovered-amid-rise-e-commerce-attacks/

@mirtchovski, I would encourage you to try and separate the deficiencies of the security industry from the needs of software engineering as a whole. The information security industry is a billion dollar industry that by in large, is filled with vaporware. The cognitive dissidence is on full display when technology is discouraged due to "what it will mean in the hands of hackers". This same mindset fueled anti-encryption sentiment in the 80s and 90s, and continues today around the need to invade user privacy more and more.

Hackers do not need this hypothetical obfuscator to lay waste to even the most sophisticated anti-malware systems. Those system's are ineffective, not because the hackers have magic tools, but because they are attempting to solve a non-sequiter. They have to be flexible enough to not cause false positive detection/prevention, but also detect nefarious behavior. Since the possible permutations of this is different not just from company to company, but from employee to employee, and even day to day per employee, heuristics based detection will never be the primary defensive strategy.

Good security is about empowering the user to be inherently secure. Go does a great job of this in their x/crypto/ssh package, as well as attempting to make certificate validation default in net/http.

Sorry for the long position, but as a security professional and an avid Go user, I've got an interest in not letting security theater discourage innovation.

[edit to add the forgotten mention to reply]

[fbnz156]

@gen0cide I believe this code generator idea to be a good workaround, and was my plan for when this change goes ahead anyway. It being part of Go's core tools would definitely soften the impact of losing BOPs. However, plugins need to work on all main platforms before they can ever be considered an alternative.

Also aside from trimpath and -s -w is there any more complete ways to remove source information from the compiled binary?

[mvdan]

I think we should focus on the technical arguments against adopting plugins. So far, it seems like the only missing critical features in plugins are Windows support and the ability to expose type definitions. Is that correct?

If so, perhaps we should move the discussion towards looking at those issues, and perhaps even blocking this proposal until they've been considered.

I don't think verbosity is nearly as important as the other issues, nor do I think it's a stopgap. Having to write an extra 50 lines is certainly not ideal, but there are many factors at play here. Perhaps Go2 type parameters would allow getting rid of some of the reflect boilerplate.

[fbnz156]

@mvdan I agree. Plugins combined with a command line tool could replace BOPs, if plugins have the OS limitation fixed, support type definitions, and provide an option to export constant values (although the command line tool could simply copy constant definitions to the shared code).

All of the boilerplate code is so generic and repetitive that it could be generated.

[rsc]

I'm not terribly worried about the "malware community" one way or another. In the page you linked, the binary hadn't even been stripped.

[rsc]

The fact is binary-only packages do _not_ work with Go 1.12 except in limited circumstances. (And I understand that some people are working within those limited circumstances successfully, if luckily.) In Go 1.13 they won't work at all. The limited cases that do work in Go 1.12 will continue to work in Go 1.12 - we aren't going to retroactively remove them from Go 1.12. People who need extra time to move to what's next can keep using Go 1.12 as long as they like, of course.

It seems like there are two paths forward. First, people could work on an obfuscator of sorts. Second, people could work on improving plugin support.

[fbnz156]

@rsc Why not wait until plugins are at a sensible point before deprecating a solution that _does_ work?

[ianlancetaylor]

@fabian-f The unfortunate fact is that it doesn't work today, as described earlier. And it's going to break even worse with future changes such as module support.

[gopherbot]

Change https://golang.org/cl/165746 mentions this issue: cmd/go: drop support for binary-only packages

[fbnz156]

@ianlancetaylor But you've heard from 3 different people who have clients who successfully use them in a production environment, so it does work, today.

[ianlancetaylor]

@fabian-f There are several options that the client could pass to go build or go install that will break the build with binary-only packages. So, yes, it works in limited circumstances.

[mikeschinkel]

Reading through all of these comments, it seems the question has evolved to _"Can plugins replace binary-only packages?"_ and _"What is needed to make binary-only packages viable?"_ Given that, I am hoping my comments are not off-topic, but if they are please forgive me and feel free to mark this comment as off-topic like one of the ones above.

We are not actually using binary-only packages but very recently explored using plugins to allow a 3rd party extension mechanism for a product we are building that would be marketed to people who are not-Go developers. IOW, we are looking for an extensibility mechanism that would allow 3rd parties to build and deliver binary "add-ons" for our product to our mutual customers. Unfortunately we found plugins lacking, for two reasons:

1. They do not work for Windows _(although I do understand that is supposed to be resolved.)_

2. Plugins evidently require being built by the same version of the compiler that the main application is built using, and that unfortunately is a non-starter given we won't always be coordinating with 3rd party developers nor would we want to have to.

So, I understand that plugins possibly — by design — will always require version parity between the main app and the plugin(s). But if that is the case then I think Go really needs a standardized extensibility mechanism that allows Go apps to load 3rd party extensions that were compiled with any compiler, maybe with communication over named pipes — can you do _(something like)_ that in Windows? — or via protobuf over TCP/IP?

We can, of course, develop our own, but I think this really should be a widely applicable use-case — imaging 3rd party extension for Docker? — and I doubt our team and many others would have the expertise and experience to envision all the edge cases that might be found by users, so I am asking that the Go team consider creating a _"blessed"_ standard package for extensibility that does not have the limitations that I understand plugins have.

That said, if I am misunderstanding anything about how plugins work, please forgive me for that as well. I have only been actively coding in Go for less than 6 months.

[shuxiao9058]

I prefer not remove the feature, for protect some source code.

[ianlancetaylor]

@shuxiao9058 Protect source code from what?

[cpipero]

I thought this whole issue was closed and the decision was made to just drop the binary packages, and maybe, possibly, hopefully improve the plugins to replace with the same value proposition (that is, it solves a hairy business issue not a technical one -- and no, I cannot force legal teams to go through sufficient computer science training to change their ways).

I understand your motivation and support your decision.

We accepted this fact, but we also accepted the fact that basically the binary-only package option was a golang failure (among many successes of course).

Our solution was to go back to good old C++ because of their stricter governance that prevents failures like this to have such short term impact.

Of course, we will use and enjoy golang whenever the customer IP is not involved in the deliverable agreement.

[robaho]

I think a big difference between binary only packages and plugins are the structs themselves. With BOP you can hide package level details in the struct, when you move to plugins, you need to create public only structs and then copy and extend to the internal structs. This is a lot of work.