Vector: NixOS: Configuration error: unknown variant `journald`

[happysalada]

In the course of providing a service for NixOS I'm running into the following error

Nov 13 00:35:44.887 ERROR vector: Configuration error: "/nix/store/qiw20ic94pzsklar8cbh336dpxj0va8j-vector.toml": unknown variant `journald`, expected one of `docker`, `file`, `generator`, `http`, `internal_metrics`, `kafka`, `logplex`, `prometheus`, `socket`, `splunk_hec`, `statsd`, `stdin`, `syslog`, `vector` for key `sources.input.type`

On NixOS vector is build with the following features enabled.

 "leveldb" "leveldb/leveldb-sys-2" "jemallocator" "rdkafka" "rdkafka/dynamic_linking"  "sinks" "sources" "transforms" 

It seems that the journald source should be included right?

I'm not sure why I'm getting the error, nor how to move forward

Vector Version

10.0

Vector Configuration File

[sinks]
[sinks.file]
inputs = ["level"]
path = "./logs/day-%d.log"
type = "file"

[sinks.file.encoding]
codec = "ndjson"

[sinks.loki]
endpoint = "http://localhost:3100"
inputs = ["level"]
type = "loki"

[sinks.loki.labels]
level = "{{ level }}"

[sources]
[sources.input]
type = "journald"

[transforms]
[transforms.level]
inputs = ["input"]
type = "lua"
version = "2"

[transforms.level.hooks]
process = "function (event, emit)\n  message = event.log.message\n  level = string.match(message, \"le?ve?l=(%w+)\")\n  msg = string.match(message, \"me?ss?a?ge?=(.+)\")\n  if level then\n    event.log.level = level\n    event.log.message = msg\n  end\n  -- Very important! Emit the processed event.\n  emit(event)\nend\n"

Expected Behavior

no errors

Actual Behavior

errors in the configuration

Example Data

Additional Context

this is on NixOS, using the default package. Perhaps the mistake is coming from how vector is built on NixOS. If I just have some ideas of where the problem could be coming from, I'm interested in testing myself.

References

Here is a reference to the PR https://github.com/NixOS/nixpkgs/pull/103393

Here is a reference of how vector is build on NixOS.
https://github.com/johanot/nixpkgs/blob/a03c8a8f35613b4ca501d09c788545d1fd7376ec/pkgs/tools/misc/vector/default.nix
(the language is Nix, but hopefully things should be fairly transparent)

[johanot]

@Hoverbear you beat me to it :D

This fixes the problem in nixpkgs: https://github.com/NixOS/nixpkgs/pull/104179 .

But perhaps the journald mod should be guarded by just #(cfg(unix)) instead of #(cfg(feature = "unix")) ? I'm not a Vector-expert, so I don't know why that feature is there?

[Hoverbear]

Ah cool here I deleted the comment thinking that wouldn't solve it! Hahah

[Hoverbear]

@Hoverbear you beat me to it :D

This fixes the problem in nixpkgs: NixOS/nixpkgs#104179 .

But perhaps the journald mod should be guarded by just #(cfg(unix)) instead of #(cfg(feature = "unix")) ? I'm not a Vector-expert, so I don't know why that feature is there?

So, it is: https://github.com/timberio/vector/blob/c357da46928b23db5016d6b5da17b44c49fb73e3/src/sources/mod.rs#L24-L25

[Hoverbear]

I don't think you should need to do this. 🤔

In https://github.com/timberio/vector/blob/c357da46928b23db5016d6b5da17b44c49fb73e3/src/sources/mod.rs#L24-L25 we see that it's guarded by #[cfg(all(unix, feature = "sources-journald"))] which is in the sources feature set: https://github.com/timberio/vector/blob/c357da46928b23db5016d6b5da17b44c49fb73e3/Cargo.toml#L311

[johanot]

Right.. It's already fixed here: https://github.com/timberio/vector/commit/9e5284a7de02f90374ddc44238484d123cac92c6 .. Didn't see that. It's just not in v0.10.0 , which is the version we have in nixos-unstable. :)

[Hoverbear]

Ah-ha! Of course. This is 0.10. We redid our release process for 0.11. https://github.com/timberio/vector/blob/a46a2e488d353eff2b476fd9984304cbe2f67424/src/sources/mod.rs#L14-L15