Ueli: Calculation is slightly off

Created on 5 Jan 2019  路  5Comments  路  Source: oliverschwendener/ueli

When you type '1.1625 - 0.2' its result is '0.9625000000000001'. It the only issue I had with calculations so far.

Screenshot:
screenshot_3

bug

All 5 comments

Actually this has to do with JavaScript having serious issues when doing math with decimals.

For example see here:

I have to look into this to find a solution. Thank you for your feedback!

Fortunately, MathJS supports BigNumber, which has better precision and eliminated JS number's problems.
We can change to BigNumber by set:

math.config({
    number: "BigNumber",
});

http://mathjs.org/docs/core/configuration.html

CalculatorInputValidator's constructor might be a good place to place this.

Fixed this issue with the help of @khanhas, thank you for your help! The fix will be shipped with the next release.

Mathjs instance we using in both CalculatorInputValidator and Searcher are the same one so you only need to set config once.
You also might want to change precision field in config since default value is really long, 64.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

FroyoXSG picture FroyoXSG  路  6Comments

moses-bowman picture moses-bowman  路  3Comments

thatKaran picture thatKaran  路  5Comments

nevara picture nevara  路  7Comments

TobiasDev picture TobiasDev  路  7Comments