Telegraf: Solaris Support

[calmh]

The CPU and terminal stuff is probably trivial, the Bolt locking thing is a long standing issue on Bolt. Do we need a database just to send values to Influx?

jb@zlogin4:~/s/g/i/telegraf $ make
go get github.com/tools/godep
godep go build -o telegraf -ldflags \
        "-X main.Version=v0.2.3-3-g2be7fc0" \
        ./cmd/telegraf/telegraf.go
# github.com/shirou/gopsutil/cpu
Godeps/_workspace/src/github.com/shirou/gopsutil/cpu/cpu.go:72: undefined: CPUTimes
Godeps/_workspace/src/github.com/shirou/gopsutil/cpu/cpu.go:73: undefined: CPUTimes
# github.com/Sirupsen/logrus
Godeps/_workspace/src/github.com/Sirupsen/logrus/text_formatter.go:28: undefined: IsTerminal
# github.com/boltdb/bolt
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:24: undefined: syscall.LOCK_SH
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:26: undefined: syscall.LOCK_EX
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:30: undefined: syscall.Flock
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:30: undefined: syscall.LOCK_NB
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:44: undefined: syscall.Flock
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:44: undefined: syscall.LOCK_UN
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:61: undefined: syscall.Mmap
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:86: undefined: syscall.Munmap
Godeps/_workspace/src/github.com/boltdb/bolt/bolt_unix.go:95: undefined: syscall.SYS_MADVISE
# github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system
Godeps/_workspace/src/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system/stat_unsupported.go:16: s.Mtimespec undefined (type *syscall.Stat_t has no field or method Mtimespec)
# github.com/fsouza/go-dockerclient/external/github.com/Sirupsen/logrus
Godeps/_workspace/src/github.com/fsouza/go-dockerclient/external/github.com/Sirupsen/logrus/text_formatter.go:28: undefined: IsTerminal
godep: go exit status 2
Makefile:11: recipe for target 'build' failed
make: *** [build] Error 1

[sparrc]

@calmh That is probably because we currently import the graphite service from influxdb to filter statsd metrics here: https://github.com/influxdb/telegraf/blob/master/plugins/statsd/statsd.go#L13

importing that service in turn imports the rest of the influxdb repo. If we moved that dependency (maybe just copy and paste the code into telegraf, or move it into an independent package within influxdb) then we wouldn't need to have all of influxdb as a dependency to telegraf.

[sparrc]

PR up that will eliminate the Bolt (and a few other) dependency #655

[sparrc]

@calmh version 0.10.3 of telegraf now has many of these dependencies removed, care to try again?

[calmh]

It's getting closer!

https://gist.github.com/41ceb6bbf11987ac935a

[psteinbachs]

I'm hitting the same issue under SmartOS. The gopsutil dependency doesn't support Solaris (and derivatives) at this time, which impacts inputs/system and inputs/procstat. It would be more effective to get these values directly from kstats. There is go-kstat which could serve as the basis for inputs/kstats in the future. In the short-term, I have excluded system and procstat which allows telegraf to compile, and intend to use the exec plugin via a shell wrapper to the kstat cli for now.

[sparrc]

I'm sure that @shirou would appreciate it if anyone sent up some PRs to support Solaris on gopsutil

[shirou]

It seems golang has solaris target. So, if anyone send me PRs with // +build solaris build option, I will accept.
And if you teach me some VPS which can use Solaris, OpenIndiana or else, I can try on it.

[calmh]

Cool. I don't have the time/inclination to hack up the required changes at the moment, but might possibly get around to it at some time. In the meantime, I have an example for CPU monitoring for Solaris here: https://github.com/syncthing/syncthing/blob/master/cmd/syncthing/gui_solaris.go (MPLv2 although I can relicense if you prefer as I'm the sole author) and can give you a SmartOS zone to run and test on if you like.

The Solaris /proc interface is clean and documented, so these things are fairly straightforward.

[jjlawren]

Has anyone been able to build on SmartOS successfully? I've forced fsnotify to compile (successfully?) to allow the github.com/influxdata/tail/watch plugin to build by removing the Solaris build mask.

However it's still failing on random plugins such as plugins/outputs/prometheus_client. What's the proper way to avoid building failing plugins?

[danielnelson]

@jjlawren Open a new issue with the compile error and we will see what we can do.

[jjlawren]

Looks like my primary problem was using a too-old version of Go in the package manager. Go 1.8 was just released in pkgsrc-2017Q1, so I'll try that first. Sorry for not digging enough by myself first.

[danielnelson]

Here are the current compile errors:

$ GOOS=solaris make telegraf
go build -ldflags " -X main.commit=9378de94 -X main.branch=master" ./cmd/telegraf
# github.com/influxdata/telegraf/vendor/github.com/karrick/godirwalk
vendor/github.com/karrick/godirwalk/readdir.go:20:9: undefined: readdirents
vendor/github.com/karrick/godirwalk/readdir.go:46:9: undefined: readdirnames
# github.com/influxdata/telegraf/vendor/github.com/docker/docker/client
vendor/github.com/docker/docker/client/client.go:238:35: undefined: DefaultDockerHost
vendor/github.com/docker/docker/client/client.go:243:12: undefined: DefaultDockerHost
vendor/github.com/docker/docker/client/client.go:247:12: undefined: defaultProto
vendor/github.com/docker/docker/client/client.go:248:12: undefined: defaultAddr

[ybinnenwegin2ip]

Solaris support was added in a newer version of Godirwalk (v1.10.1), the Docker Client error seems to occur because the go file which contains the variables that are required doesn't include Solaris.

For our use case we only want the ZFS plugin though, and that seems to be working wonderfully! I've forked Telegraf here: https://github.com/ybinnenwegin2ip/telegraf/tree/omnios_zfs

That compiles and runs on OmniOS 5.11 (omnios-r151030-f1189fc02c). I've modified the zfs_freebsd input to work on OmniOS (use kstat instead of sysctl etc). I haven't played around yet with the zpool statistics, but the general ZFS statistics end up in InfluxDB. :)

Edit:

I've now also tested the zpool input, works well! I also extended it to include more zpool-specific metrics (such as the reads/writes etc). These metrics are absent from freebsd (got no idea why) but are present in ZOL. Anyhow, adding them for OmniOS was very easy. In addition to that I've also wrote down some quick instructions on how I compile my branch on OmniOS, the document can be found in the docs folder.

[danielnelson]

I updated godirwalk to a version that supports Solaris. Remaining work is in the docker, docker_logs, and ecs input plugins.

[lachlanbb]

lachlan@singularity:~/go15/src/telegraf$ ./telegraf --version Telegraf 1.17.0 (git: HEAD 3f7a54c9) lachlan@singularity:~/go15/src/telegraf$ uname -a SunOS singularity 5.11 11.4.0.15.0 i86pc i386 i86pc

Got latest successfully compiling on 11.4... bit painful though
1) pkg install golang gcc gnu-binutils gnu-make

2) Download latest go source, compile it using standard Solaris Golang (Makefile requires 'go mod')
export GOARCH=amd64
export GOOS=solaris
GOROOT_FINAL=~/go15
Use GNU make instead of Solaris make /usr/gnu/bin/make

3) reload env, use new go
export GOPATH=~/go15
export PATH=$GOPATH/bin:$PATH

go version go1.15.6 solaris/amd64

4) Remove broken Telegraf plugins

src/plugins/inputs/all/all.go

_ "github.com/influxdata/telegraf/plugins/inputs/docker"
_ "github.com/influxdata/telegraf/plugins/inputs/docker_log"
_ "github.com/influxdata/telegraf/plugins/inputs/ecs"
_ "github.com/influxdata/telegraf/plugins/inputs/modbus"

plugins/outputs/all/all.go

_ "github.com/influxdata/telegraf/plugins/outputs/prometheus_client" # might work, not sure?

plugins/processors/all/all.go

_ "github.com/influxdata/telegraf/plugins/processors/starlark" # caused most of the issues.