Vector: Allow multiple log field type conversions

Created on 11 Dec 2019  路  1Comment  路  Source: timberio/vector

May i use one fields in multiple transform?

[transforms.some.types]
  timestamp2 = "timestamp|%Y-%m-%d %H:%M:%S%.f"
  timestamp2 = "timestamp|%d.%m.%Y %H:%M:%S"

i.e. if first transform type failed then try next
now it just dropped if failed

parsing processing nice feedback approval requirements enhancement

Most helpful comment

Hey @arrrght, unfortunately it's not possible to have duplicate keys within the types table as it is invalid TOML. However, we could potentially expand the values to support arrays, where if a conversion fails then the next is attempted, and so on.

Perhaps something like this:

[transforms.some.types]
  timestamp = [
    "timestamp|%Y-%m-%d %H:%M:%S%.f",
    "timestamp|%d.%m.%Y %H:%M:%S"
  ]

We should also capture duplicate keys and report them to the user to make it clear they're being ignored.

>All comments

Hey @arrrght, unfortunately it's not possible to have duplicate keys within the types table as it is invalid TOML. However, we could potentially expand the values to support arrays, where if a conversion fails then the next is attempted, and so on.

Perhaps something like this:

[transforms.some.types]
  timestamp = [
    "timestamp|%Y-%m-%d %H:%M:%S%.f",
    "timestamp|%d.%m.%Y %H:%M:%S"
  ]

We should also capture duplicate keys and report them to the user to make it clear they're being ignored.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

raghu999 picture raghu999  路  3Comments

a-rodin picture a-rodin  路  3Comments

binarylogic picture binarylogic  路  4Comments

LucioFranco picture LucioFranco  路  3Comments

jamtur01 picture jamtur01  路  3Comments