Vector: Should vector shutdown if the stdin source is exhausted/exited with an error?

Created on 11 Jul 2020  路  7Comments  路  Source: timberio/vector

Facing some interesting behaviour when using the stdin source with vector.
Consider this dummy setup:

sleep 5 | vector --config vector.toml

where

data_dir = "/var/lib/vector/"
dns_servers = []

[log_schema]
message_key = "message"
timestamp_key = "timestamp"
host_key = "host"

[sources.source0]
max_length = 102400
type = "stdin"

[sinks.sink0]
healthcheck = true
inputs = ["source0"]
type = "console"

[sinks.sink0.buffer]
type = "memory"
max_events = 500
when_full = "block"

(output of vector generate stdin//console).

I would expect vector to exit once the piped command exits (roughly after the 5 seconds sleep time), but it does not seem to be the case: vector will keep running indefinitely.

The same happens if the piped command exits with a non-zero exit code, e.g.:

exit 1 | vector --config vector.toml

Even using a proper bash script with pipefail does not cause vector to exit when the piped command fails, e.g.

#!/bin/bash
set -euxo pipefail

exit 1 | vector --config vector.toml

This can be fairly problematic when piping the output of an application inside vector to ship logs: we'd like to let the application crash to have it restarted by our orchestrator (e.g. Kubernetes) but vector is going to be keep the container running.
Any solution to this dilemma?

stdin help

All 7 comments

Thanks @LukeMathWalker, what you described does not sound like the intended behavior. @ktff worked on this area recently, so he'll take a look.

@LukeMathWalker I'm guessing you are using a proper release version of vector. This feature is present in the master branch, added in #2533, and will be in the next v0.10.0 release which should come out soon. @binarylogic do we have a date for it?

I am indeed running on an official release - good to know it's already fixed in master!

Yes, 0.10.0 should go out early next week. We're preparing it here.

Just chasing the last bugs! 馃悰

Great, then, @LukeMathWalker once 0.10.0 comes out could you check if it behaves as you would expect, after that we can close the issue.

0.10.0 is out now and it includes this!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

binarylogic picture binarylogic  路  3Comments

raghu999 picture raghu999  路  3Comments

LucioFranco picture LucioFranco  路  3Comments

jamtur01 picture jamtur01  路  3Comments

LucioFranco picture LucioFranco  路  3Comments