What the heck?
(pow 2 ((log10 (n-1))/(log10 2)))
32.0
32.0 as integer
32
(pow 2 ((log10 (n-1))/(log10 2))) as integer
31
Edit – After restarting the math is evaluated correctly…this is not the first time it has behaved like this, does anyone else have erratic evaluations like this?
I’m assuming it’s some kind of floating point error. I get the same thing. Perhaps something to do with the overflow?
hi,
not sure its returning the wrong value but you can use ceil to force the correct value
ceil(pow 2 ((log10 (n-1))/(log10 2))) as integer
mark
Well, the thing is that it’s not consistent. When I evaluate it now I get different results.
would you like to post the results?
mark
Welcome to the single precision world!
It IS a floating point rounding error:
n = 33
32000.0 - (pow 2.0 ((log10 (n-1))/(log10 2))) * 1000
0.0117188
As you can see, the result of your expression is slightly less than 32.0, but very very close – the difference is just around 0.0000117188…
Since AS INTEGER always rounds down,
0.9999999 as integer
0
you are getting 31 from your calculation.
To automatically round up or down depending on whether the decimal part is > 0.5 or less than 0.5, you can add 0.5 to the result and then turn to integer.
( 0.5 + (pow 2.0 ((log10 (n-1))/(log10 2))) ) as integer
If the result of a calculation was, say, 12.4, 12.4+0.5 = 12.9, and the integer would be 12. If it was 12.6, 12.6+0.5 = 13.1, so the integer would be 13, correctly rounded up.
Ok, it’s a floating point round off error I have run into that problem before. I still don’t know why it is not consistently wrong but I guess max must have some dynamic stuff in there for evaluation optimization that results in different results when it is near a round off point