Go-sqlite3: Cross compiling fails

[samhh]

Very possibly a mistake on my part, if so please let me know where I went wrong. My first time with Go today!

I have a basic Go file using this library. I am on macOS 10.12.3, Go 1.8 darwin/amd64.

Succeeds: env GOOS=darwin go build -i app.go

Fails: env GOOS=linux go build -i app.go

Specifying GOARCH=amd64 makes no difference.

Errors:

../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:18: undefined: SQLiteConn
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:26: undefined: SQLiteConn
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:27: undefined: namedValue
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:29: undefined: namedValue
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:35: undefined: SQLiteConn
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:36: undefined: namedValue
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:44: undefined: SQLiteConn
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:49: undefined: SQLiteConn
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:54: undefined: SQLiteStmt
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:63: undefined: SQLiteStmt
../../../go/src/github.com/mattn/go-sqlite3/sqlite3_go18.go:36: too many errors

Said very basic Go file causing this, in-case it helps:

package main

import (
    "database/sql"
    "log"
    "os"
    _ "github.com/mattn/go-sqlite3"
)

// Bookmark represents a single row from database
type Bookmark struct {
    Id       int
    Url      string
    Metadata string
    Tags     string
    Desc     string
    Flags    int
}

func main() {
    dbpath := GetDbPath()

    db := InitDB(dbpath)
    defer db.Close()

    bookmarks := GetAllBookmarks(db)
    log.Print(bookmarks)
}

// Write errors to log & quit
func CheckError(err error) {
    if err != nil {
        log.Fatal(err)
    }
}

func InitDB(filepath string) *sql.DB {
    db, err := sql.Open("sqlite3", filepath)
    CheckError(err)
    if db == nil {
        panic("db nil")
    }
    return db
}

func GetAllBookmarks(db *sql.DB) []Bookmark {
    const queryAll string = "SELECT * FROM bookmarks;"

    rows, err := db.Query(queryAll)
    CheckError(err)
    defer rows.Close()

    var result []Bookmark
    for rows.Next() {
        item := Bookmark{}
        err2 := rows.Scan(&item.Id, &item.Url, &item.Metadata, &item.Tags, &item.Desc, &item.Flags)
        CheckError(err2)
        result = append(result, item)
    }

    return result
}

func GetDbPath() string {
    const dbFileName string = "bookmarks.db"

    var dir = os.Getenv("XDG_DATA_HOME")
    if dir != "" {
        dir += "/buku/"
    } else {
        dir = os.Getenv("HOME") + "/.local/share/buku/"
    }

    dir += dbFileName

    return dir
}

[mattn]

Maybe you don't install/set cross C compiler for Linux.

[samhh]

Is that gcc? If so I just installed version 6 of that, but the same problem remains.

[mattn]

Not host compiler. I mean cross compiler. Install cross C compiler, then set $CC.

[samhh]

So I've installed this: https://github.com/crosstool-ng/crosstool-ng

No luck.

So I tried setting env $CC to ct-ng first, still no luck.

Unsure where to go from here.

Sorry to be a pest! 🙏

[mattn]

Unfortunately, I don't have OSX. So that's all I can tell you. Sorry

[samhh]

Well, the good news is that I can build Linux just fine if I boot an Arch VM and do it from there.

Still no idea how to get it working from within macOS, so if anyone wants to chime in that'd be appreciated. :-)

[Vorlif]

Hi Mattn,

I have the same problem with go 1.8.
If I downgrade to 1.7 there are no more problems. So I do not think there is a problem with the compiler.

OS: Linux x86_64

[mattn]

Maybe your CGO_ENABLED is 0

[Vorlif]

With go 1.7 I can cross compile without a C cross compiler (like mingw). I can use the native go compiler and have no problems.

# This works, but only with go 1.7
# Not with go 1.8
env CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build

If I understood it right CGO_ENABLED=1 requires a C cross compiler?

Is enabled cgo a new dependency if I want to use your package with go 1.8?

Thanks for your work and your time!

[mattn]

Maybe, the driver built on go1.7 does not work. It should be empty module.

[Lindenk]

I'm now having the same problem when trying to compile specifically this for ARM. It was working fine with go 1.7 for me as well.

[rusenask]

Noticed this as well, trying to create some workaround.. I am a bit sceptical that this is related to Go 1.8 as I have been using it since December and it was compiling fine. Something else changed.

[Lindenk]

Hmm....I tried backtracking and using go-sqlite3 at f043f8092b52ded4899199e875282ac4dc93d266, which was at the beginning of November with the same problem.

[Vorlif]

After checkout the revision b5c99a720374818b629fd1fbf6d2cbb4fb9a5644 (2017-08-21) the compilation is working.

[Lindenk]

Does it look like 98981b4a3bc61c9f76cd54c691f1262041973c3e is the commit that breaks cross-compiling on go 1.8?

[mattn]

@FiloSottile It seems related with your commit. Can you handle this issue? If it was temporary issue for some version of go, I'll revert the commit. Thanks.

[Vorlif]

No, I don't think that 98981b4a3bc61c9f76cd54c691f1262041973c3e is the breaking point. Firstly, the commit 5811704cf608c808d720fe1a172864cf41406efd to undo the changes. Secondly, the cross compilation does not work with both versions.

If I checkout the revision b5c99a720374818b629fd1fbf6d2cbb4fb9a5644 the cross compilation works. But the binary does not work. I'm not sure if this a problem of version mixing (go-sqlite supports only go 1.6 on this point but I use go 1.8) or a problem of the cross compilation.

@mattn Should it be possible to cross compile go-sqlite without enabled CGO?

[mattn]

Should it be possible to cross compile go-sqlite without enabled CGO?

go-sqlite3 is CGO module. So you need to enable CGO always.

[Lindenk]

Huh, I'm surprised. It seemed to be compiling and running fine until go 1.8. I'm not why then.

[Vorlif]

go-sqlite3 is CGO module. So you need to enable CGO always.

Then we can close the issue from my point of view.

I have tried to cross compile without CGO. This can not work.

[dcwangmit01]

I experienced the same problems as described by this thread, but I was eventually able to get things to work.

From a linux amd64 host, I was finally able to cross compile for linux arm. Posting below, in case it may be useful for anyone.

## Building ARM Binary
export GOOS=linux; \
export GOARCH=arm; \
export GOARM=7; \
export CC=arm-linux-gnueabihf-gcc-5; \
CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static" \
  -o "./bin/${GOOS}_${GOARCH}/the_app"

[gabrielruiu]

@dcwangmit01 what package did you install to make arm-linux-gnueabihf-gcc-5 available on your system?

[rolurq]

@gabrielruiu install gcc-5-arm-linux-gnueabihf.
That package is not available on debian 9 (stretch) so I used gcc-6-arm-linux-gnueabihf, which worked fine for me, I just had to change CC=arm-linux-gnueabihf-gcc-6.

[gertcuykens]

Can't find the correct compiler arguments for OSX to linux using the clang compiler, help greatly appreciated thx

% env GOOS=linux CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static"
# runtime/cgo
ld: unknown option: --build-id=none
clang: error: linker command failed with exit code 1 (use -v to see invocation)

% env GOOS=linux CGO_ENABLED=1 go build
# runtime/cgo
ld: unknown option: --build-id=none
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[bachmaierniklas]

@gertcuykens did you find out what are the correct compiler arguments for OSX?

[gertcuykens]

No ended up using another linux docker container and compile it in that docker container instead. Sad that c compilers still make life hard even today for just a few parameters

[cosmosinnovate]

I have the same problem. I can see errors.

[mattn]

What is bad? sqlite3 is written in C code. so You MUST use CGO.

[carosatig]

I succeeded to cross compile from macos to linux using musl-cross.
brew install FiloSottile/musl-cross/musl-cross

using this command line:

CC=x86_64-linux-musl-gcc CXX=x86_64-linux-musl-g++ GOARCH=amd64 GOOS=linux CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static"

Hope this helps others.

[matthewmueller]

@carosatig just what I was looking for! thanks for figuring that out for us.

i'm not too familiar with the cgo compilation process, do you mind explaining why -ldflags "-linkmode external -extldflags -static" is needed?

[sungwoncho]

I am cross compiling using xgo. Here is my working solution https://github.com/dnote/cli/blob/fa1da50fc5f8f9ef9df23a35a86e497ed9a9bbff/scripts/build.sh#L31-L60 in case it helps anyone.

[MicroIdea]

I succeeded to cross compile from macos to linux using musl-cross.
brew install FiloSottile/musl-cross/musl-cross

using this command line:

CC=x86_64-linux-musl-gcc CXX=x86_64-linux-musl-g++ GOARCH=amd64 GOOS=linux CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static"

Hope this helps others.

THANKS! Really saved my day~
For those who want to compile ARM like Raspberry pi:
brew install FiloSottile/musl-cross/musl-cross --without-x86_64 --with-arm-hf
You can also use --with-i486 (x86 32-bit), --with-aarch64 (ARM 64-bit), --with-arm (ARM soft-float) and --with-mips.
But it takes a lot of time: on my old Mac built in 174 minutes 44 seconds
BTW, my build command is CC=arm-linux-musleabihf-gcc CXX=arm-linux-musleabihf-g++ GOARCH=arm GOARM=7 GOOS=linux CGO_ENABLED=1 go build -ldflags "-linkmode external -extldflags -static" xxx.go

[hongjinlin]

CC=mips-linux-musl-gcc CXX=mips-linux-musl-g++ CGO_ENABLED=1 GOOS=linux GOARCH=mipsle GOMIPS=softfloat go build -ldflags="-s -w" main.go

How can i build for little endian?Thank you!
If i change "GOARCH=mipsle" To "GOARCH=mips" is ok, but my target is little endian~

[rittneje]

You should probably open a new issue rather than commenting on one that’s already been closed. Please include the actual problem you’ve encountered. If it’s failing to compile, please include any error messages.

[hongjinlin]

You should probably open a new issue rather than commenting on one that’s already been closed. Please include the actual problem you’ve encountered. If it’s failing to compile, please include any error messages.

ok , thank you!

[sosoyososo]

compiling for linux on macOS can refer this solution #797

[vishnukvmd]

@carosatig 's comment was quite helpful in resolving compilation issues on OSX. I'd recommend documenting this.

@mattn Please let me know if you would be okay with me creating a PR that appends this information to README.md.

[mattn]

Yes, please.

[vishnukvmd]

@mattn Here you go: https://github.com/mattn/go-sqlite3/pull/804

[peterwillcn]

Good