Influxdb: For float fields, if we receive an INT it should be cast to float64. (see paul's comment below)

Created on 18 Jun 2015  路  12Comments  路  Source: influxdata/influxdb

the value type for a metric is stored as decimal (money 50.99). Sometimes the value to write will be a whole number (60).

Influxdb will respond with BadRequestError: write failed: field type conflict: input field "value" is type int64, already exists as type %!s(influxql.DataType=1)

In javascript, even if I explicitly set a number variable as 60.0, it is set as 60. I can easily enough convert toString / toFixed(2) on write, but I would hope that if the metric field type exists as decimal, the database would implicitly save a whole number (60) as (60.0). Thoughts?

RFC

Most helpful comment

@j-rock what if the user expected to be writing floats, but the very first value was a whole number so it gets set to ints. From that point forward we'll be dropping precision while the user expects us to be keeping it.

This whole int/float thing is making me think that it might be a good idea to force everything to be a float like Prometheus does.

All 12 comments

i can completely feel the pain from the JS side of things. if it's not a burden to do this within the db i agree it would be helpful. maybe the parser for the line protocol could treat all numeric values as floats? interacting more directly with the go api would still allow for other numeric types.

This would likely be mitigated by a fix for #3006, but that's not going to happen in the short term.

In the short term, perhaps we can automatically cast incoming int64 as float64 if the underlying target field value is already cast as a float64. The only downside is int64s that exceed float64s capacity, but that's an unlikely edge case.

@pauldix @corylanou @benbjohnson, thoughts?

Has there been any progress on this or #3006? Inquiring minds would love to know.

yeah, I think automatic casing should be done. However, we should catch an overflow error and return an error if that occurs.

This seems like a good usability enhancement so I've renamed the issue and assigned it to the 0.9.3 release.

However, I will say that this can create a problem if the first value you write is a whole number and future values are floats. That will result in an error.

Yes. Catching overflow errors on int64 -> float64 sounds reasonable. Why not truncate a float64 to an int64 if the series expects int64?

@j-rock what if the user expected to be writing floats, but the very first value was a whole number so it gets set to ints. From that point forward we'll be dropping precision while the user expects us to be keeping it.

This whole int/float thing is making me think that it might be a good idea to force everything to be a float like Prometheus does.

@pauldix presumably we would keep time as int64, and just reduce field values to boolean, float64, and string?

What's the math performance hit for doing floats vs ints?

@beckettsean not sure, would require testing. Although for it to really matter they'd have to be churning through millions of points.

What if choosing only [boolean, string, float64] as field values was a config parameter?

from a coding/support standpoint that won't make our job any easier. The goal with forcing float64 is that it makes everything simpler and makes it so the user is less likely to shoot themselves in the foot.

all numbers are now floats unless explicitly declared integers during writes (with a trailing i #3526). Closing as no longer relevant.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

allenbunny picture allenbunny  路  3Comments

binary0111 picture binary0111  路  3Comments

jayannah picture jayannah  路  3Comments

deepujain picture deepujain  路  3Comments

airyland picture airyland  路  3Comments