weird number calculating

John Nilsson | 2 Nov 2008 14:19

Re: weird number calculating

In principle one would think that correctness should have priority over performance. To that end, wouldn't it be better if the default decimal type was BigDecimal and default integer type BigInt (or rather something similar to them).

If you really want rounding errors, overflows, integer division and other idiosyncrasies in the name of performance couldn't that be achieved by means of implicits, or import of some kind?

Of course you would need some way to interface with Java libraries with these types. But it should be easy to handle most cases by autoboxing.

BR,
John

On Sun, Nov 2, 2008 at 5:58 AM, Sean McDirmid <sean.mcdirmid <at> gmail.com> wrote:

In general you have to put up with the imprecision of floats and
doubles. If you want precision, Scala offers big num support. However,
you would be sacrificing performance.

If this is really surprising, its time to brush up on your computer
architecture :) More links:
http://en.wikipedia.org/wiki/Floating_point_unit
http://en.wikipedia.org/wiki/IEEE_floating-point_standard
http://en.wikipedia.org/wiki/Pentium_FDIV_bug (old but still fun!)

On Sat, Nov 1, 2008 at 9:31 PM, Alex Boisvert <boisvert <at> intalio.com> wrote:
> Some background on this:
> http://www.ibm.com/developerworks/java/library/j-jtp0114/
>
> And a good rant on Java floating points if you have some extra time on your
> hands:
> http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf
>
> alex
>
> 2008/11/1 gerry xiao <gerryxiao <at> gmail.com>
>>
>> scala> 1*0.1
>> res30: Double = 0.1
>>
>> scala> 2*0.1
>> res31: Double = 0.2
>>
>> scala> 3*0.1
>> res32: Double = 0.30000000000000004
>>
>> scala> 3.0*0.1
>> res33: Double = 0.30000000000000004
>>
>> scala> 5.0*0.1
>> res34: Double = 0.5
>>
>> scala> 6*0.1
>> res35: Double = 0.6000000000000001
>>
>> what happened to 3n * 0.d?
>
>