Page 1 of 1

Float value inaccuracies - how to trace?

Posted: Thu Apr 21, 2016 6:45 pm
by treii28
I'm using combinations of Python, php and sqlite to read, store and replace floating point values. I'm aware of some of the inaccuracy issues in PHP but haven't really noticed them as profoundly as when I move scripts from other platforms to the RPi.

For example, reading i2c sensor values with i2c get via a shell_exec in php, then storing to a REAL column in an sqlite3 file, then retrieving the values again using adminer.php, I end up with something similar to this:

Modify picoStatusId timestamp mode batlevel rpilevel ad1v ad2v cputemp tempC humiP sot23temp to92temp
edit 1532 2016-04-21_13:49:20 1 2.4599999999999999645 4.7800000000000002487 0.51000000000000000888 0.68999999999999994671 30.399999999999998579 20 37 33 0
edit 1531 2016-04-21_13:48:05 1 2.4599999999999999645 4.7800000000000002487 0.54000000000000003553 0.68999999999999994671 30.899999999999998579 20 37 33 0

So, where are those long-winded decimals most likely being plopped in and how can I find out? I don't have any handy tools for staring directly into the sqlite3 file to see if that's how the values got there and I'm using PHP to get them in and out. So tracing the direct source of where the noise is coming in is hard to do when two of the tools are always involved in any check.

Re: Float value inaccuracies - how to trace?

Posted: Fri Apr 22, 2016 9:14 am
by jojopi
Floating-point/real types cannot represent decimal fractions exactly, because one-tenth has infinitely repeating digits after the binary point.

In your case I do not see that this is a problem. The errors are eighteen digits down, and your temperature measurements will neither be sensitive nor accurate down to the attokelvins.

When you come to print the values, use PHP printf() or Python str.format() to restrict the number of significant figures or decimal places.

Code: Select all

php > printf("%.1f\n", 30.3999999);

python >>> print("{:.1f}".format(30.3999999))

Re: Float value inaccuracies - how to trace?

Posted: Fri Apr 22, 2016 10:05 am
by jahboater
I don't know how your floats are being stored, but x86 machines may use 80-bits precision (64 bit significand), the ARM/VFP/NEON will only be 64-bits (53-bit significand).

Re: Float value inaccuracies - how to trace?

Posted: Fri Apr 22, 2016 10:27 am
by RaTTuS
personally I'd convert input floats to ints for temp
[or temp*10] as anything <0.1deg is not really relevant [well even 1Deg is probably good enough]