Oj: Oj rounding inconsistency

[ericmwalsh]

irb(main):055:0> Oj.dump(81.9)
=> "81.90000000000001"
irb(main):056:0> Oj.dump(81.8)
=> "81.8"
irb(main):057:0> Oj.dump(81.7)
=> "81.7"
irb(main):058:0> Oj.dump(81.6)
=> "81.59999999999999"
irb(main):059:0> Oj.dump(81.5)
=> "81.5"
irb(main):060:0> Oj.dump(81.4)
=> "81.40000000000001"
irb(main):061:0> Oj.dump(81.3)
=> "81.3"
irb(main):062:0> Oj.dump(81.2)
=> "81.2"
irb(main):063:0> Oj.dump(81.1)
=> "81.09999999999999"
irb(main):064:0> Oj.dump(81.0)
=> "81.0"
irb(main):065:0> Oj.dump(80.9)
=> "80.90000000000001"
irb(main):066:0> Oj.dump(80.8)
=> "80.8"
irb(main):067:0>

This is using the latest version of Oj - is there any reason Oj does this? I could only find a very old issue regarding a similar regarding extra decimals being added and it seemed like more of a Rails configuration issue.

[ohler55]

What mode are you using? What you are seeing is roundoff variations. Its not that extra decimals are being added but that floats are only good for 15 to 17 digits depending on the number. Ruby has a very large chunk of code that normalizes the numbers. Oj, when using anything other than the Ruby formatting uses standard C but for long doubles. That only helps a little as the numbers still have to come from Ruby. Probably more than you wanted to hear.

[ericmwalsh]

irb(main):097:0> Oj.dump(8.3, mode: :compat)
=> "8.300000000000001"
irb(main):098:0> Oj.dump(8.3, mode: :wab)
=> "8.300000000000001"
irb(main):099:0> Oj.dump(8.3, mode: :strict)
=> "8.300000000000001"
irb(main):100:0> Oj.dump(8.3, mode: :object)
=> "8.300000000000001"
irb(main):101:0> Oj.dump(8.3, mode: :null)
=> "8.300000000000001"
irb(main):102:0> Oj.dump(8.3, mode: :custom)
=> "8.300000000000001"
irb(main):103:0> Oj.dump(8.3, mode: :rails)
=> "8.300000000000001"
irb(main):104:0> 

[ohler55]

This is odd, I just ran the same tests here with Ruby 2.4.1 on OS X and in all modes the result was '8.3'. Digging further, what OS and what Ruby version are you using?

[ericmwalsh]

OS X and 2.4.1 as well

[ohler55]

Ok, lets look at the default options. I reran, I had missed the wab mode before but here are my results.

irb -Iext -Ilib -roj
irb(main):001:0> Oj.dump(8.3)
=> "8.3"
irb(main):002:0> Oj.dump(8.3, mode: :compat)
=> "8.3"
irb(main):003:0> Oj.dump(8.3, mode: :wab)   
=> "8.300000000000001"
irb(main):004:0> Oj.dump(8.3, mode: :strict)
=> "8.3"
irb(main):005:0> Oj.dump(8.3, mode: :object)
=> "8.3"
irb(main):006:0> Oj.dump(8.3, mode: :null)  
=> "8.3"
irb(main):007:0> Oj.dump(8.3, mode: :custom)
=> "8.3"
irb(main):008:0> Oj.dump(8.3, mode: :rails) 
=> "8.3"
irb(main):009:0> Oj.default_options.each_pair {|k,v| puts "#{k}: #{v}"}
indent: 0
second_precision: 9
circular: false
class_cache: true
auto_define: false
symbol_keys: false
bigdecimal_as_decimal: true
use_to_json: false
use_to_hash: false
use_as_json: false
nilnil: false
empty_string: true
allow_gc: true
quirks_mode: true
allow_invalid_unicode: false
allow_nan: true
float_precision: 16
mode: object
escape_mode: json
time_format: unix
bigdecimal_load: auto
create_id: json_class
space: 
space_before: 
object_nl: 
array_nl: 
nan: auto
omit_nil: false
hash_class: 
array_class: 

[ericmwalsh]

So I just noticed that this is only happening when I test using rails c. My results are identical to yours in irb

$ irb -Iext -Ilib -roj
irb(main):001:0> Oj.dump(8.3)
=> "8.3"
irb(main):002:0> Oj.dump(8.3, mode: :compat)
=> "8.3"
irb(main):003:0> Oj.dump(8.3, mode: :wab) 
=> "8.300000000000001"
irb(main):004:0> Oj.dump(8.3, mode: :strict)
=> "8.3"
irb(main):005:0> Oj.dump(8.3, mode: :object)
=> "8.3"
irb(main):006:0> Oj.dump(8.3, mode: :null)
=> "8.3"
irb(main):007:0> Oj.dump(8.3, mode: :custom)
=> "8.3"
irb(main):008:0> Oj.dump(8.3, mode: :rails)
=> "8.3"
irb(main):009:0> Oj.default_options.each_pair {|k,v| puts "#{k}: #{v}"}
indent: 0
second_precision: 9
circular: false
class_cache: true
auto_define: false
symbol_keys: false
bigdecimal_as_decimal: true
use_to_json: false
use_to_hash: false
use_as_json: false
nilnil: false
empty_string: true
allow_gc: true
quirks_mode: true
allow_invalid_unicode: false
allow_nan: true
float_precision: 16
mode: object
escape_mode: json
time_format: unix
bigdecimal_load: auto
create_id: json_class
space: 
space_before: 
object_nl: 
array_nl: 
nan: auto
omit_nil: false
hash_class: 
array_class:

[ericmwalsh]

$ bundle exec rails c
Running via Spring preloader in process 4375
Loading development environment (Rails 5.0.2)
irb(main):001:0> Oj.dump(8.3)
=> "8.300000000000001"
irb(main):002:0> Oj.dump(8.3, mode: :compat)
=> "8.300000000000001"
irb(main):003:0> Oj.dump(8.3, mode: :wab)
=> "8.300000000000001"
irb(main):004:0> Oj.dump(8.3, mode: :strict)
=> "8.300000000000001"
irb(main):005:0> Oj.dump(8.3, mode: :object)
=> "8.300000000000001"
irb(main):006:0> Oj.dump(8.3, mode: :null)
=> "8.300000000000001"
irb(main):007:0> Oj.dump(8.3, mode: :custom)
=> "8.300000000000001"
irb(main):008:0> Oj.dump(8.3, mode: :rails)
=> "8.300000000000001"
irb(main):009:0> Oj.default_options.each_pair {|k,v| puts "#{k}: #{v}"}
indent: 0
second_precision: 3
circular: false
class_cache: false
auto_define: false
symbol_keys: false
bigdecimal_as_decimal: false
use_to_json: true
use_to_hash: false
use_as_json: false
nilnil: false
empty_string: true
allow_gc: true
quirks_mode: true
allow_invalid_unicode: false
allow_nan: false
float_precision: 16
mode: compat
escape_mode: unicode_xss
time_format: ruby
bigdecimal_load: float
create_id: json_class
space: 
space_before: 
object_nl: 
array_nl: 
nan: raise
omit_nil: false
hash_class: 
array_class: 

[ohler55]

Try this 8.3.to_json.

That is what the default you have are calling or at least I think they are. In which case that is Rails making string. If you were running Oj mimicking Rails I could be Oj but from the looks of it you are using Rails. You could also try setting the :use_to_json to false.

[ericmwalsh]

This is via rails console:

irb(main):119:0> Oj.dump(
irb(main):120:1*   8.3,
irb(main):121:1*   {
irb(main):122:2*     second_precision: 9,
irb(main):123:2*     class_cache: true,
irb(main):124:2*     bigdecimal_as_decimal: true,
irb(main):125:2*     use_to_json: false,
irb(main):126:2*     allow_nan: true,
irb(main):127:2*     mode: :object,
irb(main):128:2*     escape_mode: :json,
irb(main):129:2*     time_format: :unix,
irb(main):130:2*     bigdecimal_load: :auto,
irb(main):131:2*     nan: :auto
irb(main):132:2>   }
irb(main):133:1> )
=> "8.300000000000001"
irb(main):134:0> 8.3.to_json
=> "8.300000000000001"

The options I passed in are the discrepancies that appeared between irb / bundle exec rails c in Oj.default_options

[ohler55]

You should probably use strict mode or something other than object but that should not make the difference as we saw it work fine outside of rails. I tried in a rails console here and 8.3.to_json came out as 8.3. I might have found a difference. Are you calling Oj.optimize_rails? I tried that and it not gives the precision issue results. That I'll check on but is it what you are seeing?

[ericmwalsh]

I have an initializer for the rails environment that is calling Oj.optimize_rails so it should be called by default.

Here is an example with me calling it explicitly then attempting to dump in strict mode:

irb(main):141:0> Oj.optimize_rails()
=> nil
irb(main):142:0> Oj.dump(8.3,mode: :strict)
=> "8.300000000000001"

[ohler55]

I'm not sure why that is happening strict mode but will look into it. I know dropping into the rails mode does make some global changes.

[ericmwalsh]

Thanks @ohler55, I appreciate your fast responses and support.

[ohler55]

Take a look at the precision branch and see if that works better for you.

[ericmwalsh]

irb(main):009:0> Oj::VERSION
=> "3.3.8"
irb(main):010:0> Oj.dump(8.3, mode: :object)
=> "8.300000000000001"
irb(main):011:0> Oj.dump(8.3, mode: :strict)
=> "8.300000000000001"
irb(main):012:0> Oj.dump(8.3, mode: :compat)
=> "8.300000000000001"
irb(main):013:0> Oj.dump(8.3, mode: :null)
=> "8.300000000000001"
irb(main):014:0> Oj.dump(8.3, mode: :custom)
=> "8.300000000000001"
irb(main):015:0> Oj.dump(8.3, mode: :rails)
=> "8.3"
irb(main):016:0> Oj.dump(8.3, mode: :wab)
=> "8.300000000000001"

[ericmwalsh]

Interestingly Oj.optimize_rails() is not setting the mode to :rails (at least for me)...

irb(main):001:0> Oj.dump(8.3)
=> "8.300000000000001"
irb(main):002:0> Oj.optimize_rails()
=> nil
irb(main):003:0> Oj.dump(8.3)
=> "8.300000000000001"
irb(main):004:0> Oj.default_options[:mode]
=> :compat

[ohler55]

Odd, I guess I still have work to do. I have to figure out why our environments are different.

[ohler55]

I gave up trying to figure out what could be different and just assumed all would have the problem. I just pushed another attempt.

[ericmwalsh]

Hmm now none of them work, mode: :rails included:

irb(main):007:0> Oj.dump(8.3, mode: :object)
=> "8.300000000000001"
irb(main):008:0> Oj.dump(8.3, mode: :strict)
=> "8.300000000000001"
irb(main):009:0> Oj.dump(8.3, mode: :compat)
=> "8.300000000000001"
irb(main):010:0> Oj.dump(8.3, mode: :null)
=> "8.300000000000001"
irb(main):011:0> Oj.dump(8.3, mode: :custom)
=> "8.300000000000001"
irb(main):012:0> Oj.dump(8.3, mode: :rails)
=> "8.300000000000001"
irb(main):014:0> Oj.dump(8.3, mode: :wab)
=> "8.300000000000001"

[ohler55]

Are you using the precision branch? We can't be that far off.

[ericmwalsh]

@ohler55 my apologies, had an issue with my rbenv:

based off your latest changes:

irb(main):001:0> Oj::VERSION
=> "3.3.8"
irb(main):002:0> Oj.dump(8.3)
=> "8.3"
irb(main):006:0> Oj.default_options[:mode]
=> :compat
irb(main):007:0> Oj.dump(8.3, mode: :object)
=> "8.3"
irb(main):008:0> Oj.dump(8.3, mode: :strict)
=> "8.3"
irb(main):010:0> Oj.dump(8.3, mode: :compat)
=> "8.3"
irb(main):011:0> Oj.dump(8.3, mode: :null)
=> "8.3"
irb(main):012:0> Oj.dump(8.3, mode: :custom)
=> "8.3"
irb(main):013:0> Oj.dump(8.3, mode: :rails)
=> "8.3"
irb(main):014:0> Oj.dump(8.3, mode: :wab)
=> "8.300000000000001"

[ohler55]

wab is fine. It is intended to be the fastest of the bunch and not care about cosmetics such as precision when it is outside the range of what is supported by the IEEE standard. Precisions above 15 decimals is not guaranteed so wab doesn't bother with the rounding since the default is 15 places and not 16.

[ericmwalsh]

If that is the case it looks like the changes are good. Thanks for your time and support @ohler55, I very much appreciate it.

[ohler55]

Have to support my users. 😄

[ericmwalsh]

@ohler55 going to close this issue, thanks again.