Go-ipfs: Support WASM Architecture

[magik6k]

So Go now Supports targeting WASM, but with certain limitations (like not having 'normal' syscalls). It's very low priority and probably a bit of work, but would be a nice 'benchmark' of portability.

First step would be to make the thing compile at all, currently stack of errors I get is down to:

# gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:23:14: undefined: haveArmSha
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:27:16: undefined: cpuid
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:32:13: undefined: xgetbv
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:40:18: undefined: cpuid
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:44:19: undefined: cpuidex
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:52:18: undefined: cpuid
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:56:17: undefined: cpuid
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:61:14: undefined: xgetbv
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:62:20: undefined: cpuidex
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:90:16: undefined: cpuid
../../../gx/ipfs/QmXTpwq2AkzQsPjKqFQDNY2bMdsAT53hUBETeyj8QRHTZU/sha256-simd/cpuid.go:90:16: too many errors
# gx/ipfs/QmPtj12fdwuAqj9sBSTNUxBNu8kCGNp8b3o8yUzMm5GHpq/pb
../../../gx/ipfs/QmPtj12fdwuAqj9sBSTNUxBNu8kCGNp8b3o8yUzMm5GHpq/pb/pb.go:421:9: undefined: terminalWidth
../../../gx/ipfs/QmPtj12fdwuAqj9sBSTNUxBNu8kCGNp8b3o8yUzMm5GHpq/pb/pb.go:430:18: undefined: terminalWidth
# gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:33:9: undefined: available
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:50:6: undefined: available
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:54:9: undefined: listenStream
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:61:6: undefined: available
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:65:9: undefined: listenPacket
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:72:6: undefined: available
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:102:6: undefined: available
../../../gx/ipfs/QmSvfeW68LC13nVt3BiwHKFneSa4DCdq3erG8RNtJvq7Ni/go-reuseport/interface.go:106:9: undefined: dial

Built with GOOS=js GOARCH=wasm go build -tags="nofuse purego" -ldflags="-X "github.com/ipfs/go-ipfs".CurrentCommit=47e9466ac" -o "cmd/ipfs/ipfs" "github.com/ipfs/go-ipfs/cmd/ipfs"

• sha256-simd - fixed in https://github.com/minio/sha256-simd/pull/33, so an update will fix those
• go-reuseport - we should find a way to just disable it, probably by either just plugin-ifying transports or sprinkling build tags injecting dummy things in place of the implementation. Probably a good excuse to fix some technical debt.
• pb (progress bar) - no idea, probably just PR a fix/workaround

Next step would be to write some transport to talk to outside nodes. /ws//wss should be easy to wrap.

[terorie]

https://github.com/cheggaaa/pb/pull/140 should fix pb wasm support
EDIT: Merged 😃

[magik6k]

Update:

With https://github.com/ipfs/go-ipfs/pull/5871, https://github.com/ipfs/go-ipfs/pull/5872, https://github.com/libp2p/go-reuseport/pull/54 and everything in plugin/loader/preload_list commented out, calling:

GOPATH/github.com/ipfs/go-ipfs/cmd/ipfs$ GOOS=js GOARCH=wasm go run -tags="nofuse purego" -ldflags="-X "github.com/ipfs/go-ipfs".CurrentCommit=47e9466ac" -exec="$(go env GOROOT)/misc/wasm/go_js_wasm_exec" . swarm peers

Gets to:

panic: JavaScript error: undefined is not a constructor

goroutine 16 [running]:
syscall/js.Value.New(0x7ff8000000000001, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/syscall/js/js.go:313 +0x7
net/http.(*Transport).RoundTrip(0x12b9100, 0xc0d9500, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/http/roundtrip_js.go:66 +0x18
net/http.send(0xc0d9500, 0x484c60, 0x12b9100, 0x0, 0x0, 0x0, 0xc00c628, 0x0, 0xc22f8f0, 0x1)
        /usr/lib/go/src/net/http/client.go:250 +0x16
net/http.(*Client).send(0x12e52c0, 0xc0d9500, 0x0, 0x0, 0x0, 0xc00c628, 0x0, 0x1, 0x0)
        /usr/lib/go/src/net/http/client.go:174 +0x10
net/http.(*Client).do(0x12e52c0, 0xc0d9500, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/http/client.go:641 +0x17
net/http.(*Client).Do(0x12e52c0, 0xc0d9500, 0xc0d9500, 0x0, 0x0)
        /usr/lib/go/src/net/http/client.go:509 +0x2
gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/http.(*client).Send(0xc22ccc0, 0xc181b20, 0x0, 0x0, 0x0, 0x0)
        /home/magik6k/.opt/go/src/gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/http/client.go:177 +0xc
gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/http.(*client).Execute(0xc22ccc0, 0xc181b20, 0x4928c0, 0xc181c70, 0x483e60, 0xc234300, 0x0, 0x0)
        /home/magik6k/.opt/go/src/gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/http/client.go:85 +0x5
gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/cli.Run.func4(0x2d8d7668, 0xc22ccc0, 0xc181b20, 0x4928c0, 0xc181c70, 0x483e60, 0xc234300, 0xc234420)
        /home/magik6k/.opt/go/src/gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/cli/run.go:145 +0x3
created by gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/cli.Run
        /home/magik6k/.opt/go/src/gx/ipfs/QmaAP56JAwdjwisPTu4yx17whcjTr6y5JCSCF77Y1rahWV/go-ipfs-cmds/cli/run.go:144 +0x36
exit status 2

So it builds! And even reads the api file! (very slowly, probably because the generated wasm binary is 39M)

From this point it should be possible to start creating a wrapper for the constructor exposing more useful things to JS so it's possible to interface with this in browser, then implement datastore backed by some browser storage and libp2p transport wrapping js websockets.

[magik6k]

Some progress has been made in https://github.com/ipfs/go-ipfs/pull/5878 - go-ipfs now runs in browsers and talks to the outside!