Tracing Floats

[Ats]

Hello. I made a little function a while ago to simply trace floats. But I just discovered it crashes if the number I want to display is lower than 1.

Code: Select all

void traceFloat(float n, byte d) {
  string integer = intToStr(n);
  string dust = subStr(intToStr(n * pow(10, d)), length(integer), d);
  trace(integer + "." + dust);
}

So I rewrote it like that:

Code: Select all

<?xml version="1.0" encoding="iso-8859-1" ?>
<ZApplication Name="App" Caption="ZGameEditor application" FileVersion="2">
  <OnLoaded>
    <ZLibrary>
      <Source>
<![CDATA[void traceFloat(float n, byte d) {
  if (d>8) d = 8;
  int integer = n;
  string str_integer = intToStr(integer);
  float decimals = n - integer;
  int dec2int = abs(decimals * pow(10, d));
  string str_decimals = intToStr(dec2int);
  trace(str_integer + "." + str_decimals);
}]]>
      </Source>
    </ZLibrary>
    <ZExpression>
      <Expression>
<![CDATA[trace("");
traceFloat(1000.123456789, 5);
traceFloat(100.123456789, 5);
traceFloat(0.123456789, 5);
trace("");]]>
      </Expression>
    </ZExpression>
  </OnLoaded>
</ZApplication>

But depending on the length of the number, not counting the decimals, and on the number of decimals I want to display, the result varies... a lot.
I guess it is some kind of limitation with int or float size, but I don't understand why...?
Also, there may be a better way to do that

[Kjell]

Hi Ats,

I guess it is some kind of limitation with int or float size, but I don't understand why...?

A signed 32-bit integer has a range of -2147483648 to 2147483647. Which means you can safely multiply the fractional part of a float by ( up to ) 10⁸ .. so that's all fine. However, a 32-bit float uses 1-bit to determine the sign of the number, 8-bits to determine the range in which the number resides, and 23-bits to determine where in that range the number resides ( as a fixed-point using a resolution of 2²³ = 8388608 ). So for instance the closest representation of 1000.123456789 is actually 1000.12347412109375.



- The sign bit is unset which means the number is positive.
- The 8 exponent bits are set to 136, which means 2⁹, which represents a range from 512 to 1024.
- The 23 fraction bits are set to 7997415, which basically results in 512+512*(7997415/8388608) = 1000.12347412109375

+ Just for good measure & posterity .. proof that this is indeed the closest representation of 1000.123456789:

Code: Select all

┌──────────┬─────────────────────┬──────────────────┐
│ Fraction │ Value               │ Inaccuracy       │
├──────────┼─────────────────────┼──────────────────┤
│ 7997414  │ 1000.1234130859375  │ 0.0000437030625  │
│ 7997415  │ 1000.12347412109375 │ 0.00001733209375 │
│ 7997416  │ 1000.12353515625    │ 0.00007836725    │
└──────────┴─────────────────────┴──────────────────┘

K

[Ats]

Oh, all right. Thanks for the explanations, Kjell