Posts: 97
Joined: Fri May 10, 2013 4:52 pm

Float value inaccuracies - how to trace?

Thu Apr 21, 2016 6:45 pm

I'm using combinations of Python, php and sqlite to read, store and replace floating point values. I'm aware of some of the inaccuracy issues in PHP but haven't really noticed them as profoundly as when I move scripts from other platforms to the RPi.

For example, reading i2c sensor values with i2c get via a shell_exec in php, then storing to a REAL column in an sqlite3 file, then retrieving the values again using adminer.php, I end up with something similar to this:

Modify picoStatusId timestamp mode batlevel rpilevel ad1v ad2v cputemp tempC humiP sot23temp to92temp
edit 1532 2016-04-21_13:49:20 1 2.4599999999999999645 4.7800000000000002487 0.51000000000000000888 0.68999999999999994671 30.399999999999998579 20 37 33 0
edit 1531 2016-04-21_13:48:05 1 2.4599999999999999645 4.7800000000000002487 0.54000000000000003553 0.68999999999999994671 30.899999999999998579 20 37 33 0

So, where are those long-winded decimals most likely being plopped in and how can I find out? I don't have any handy tools for staring directly into the sqlite3 file to see if that's how the values got there and I'm using PHP to get them in and out. So tracing the direct source of where the noise is coming in is hard to do when two of the tools are always involved in any check.

User avatar
Posts: 3230
Joined: Tue Oct 11, 2011 8:38 pm

Re: Float value inaccuracies - how to trace?

Fri Apr 22, 2016 9:14 am

Floating-point/real types cannot represent decimal fractions exactly, because one-tenth has infinitely repeating digits after the binary point.

In your case I do not see that this is a problem. The errors are eighteen digits down, and your temperature measurements will neither be sensitive nor accurate down to the attokelvins.

When you come to print the values, use PHP printf() or Python str.format() to restrict the number of significant figures or decimal places.

Code: Select all

php > printf("%.1f\n", 30.3999999);

python >>> print("{:.1f}".format(30.3999999))

Posts: 5671
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: Float value inaccuracies - how to trace?

Fri Apr 22, 2016 10:05 am

I don't know how your floats are being stored, but x86 machines may use 80-bits precision (64 bit significand), the ARM/VFP/NEON will only be 64-bits (53-bit significand).
Pi4 8GB running PIOS64

User avatar
Posts: 10558
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Float value inaccuracies - how to trace?

Fri Apr 22, 2016 10:27 am

personally I'd convert input floats to ints for temp
[or temp*10] as anything <0.1deg is not really relevant [well even 1Deg is probably good enough]
How To ask Questions :-
WARNING - some parts of this post may be erroneous YMMV


Return to “Troubleshooting”