User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 2:05 pm

Heater wrote:
Sat Jun 19, 2021 11:35 am
cppreference.com tells me :
undefined behavior - there are no restrictions on the behavior of the program. Examples of undefined behavior are data races, memory accesses outside of array bounds, signed integer overflow....
Yes, that's true now. But it wont be in time. Your link above says:-
[P0907r1] stands to change C++20 and make two’s complement the only signed integer representation that is supported.
If that change is accepted, then signed integer overflow will stop being UB from C++20
(Because with only one representation supported, there is only one possible result for overflow, therefore it becomes "defined").
Heater wrote:
Sat Jun 19, 2021 11:35 am
If ones code behaves differently, or can behave differently in some situations, when a magic compiler switch is used, then clearly one cannot deduce the behaviour of the code from the source. Which is the thing that niggles me.

If ones code always behaves the same then of course there is no need for the magic switch.
If the magic switch simply makes it go faster then I would argue that the behavior is the same.
Similar to how the "as if" rule excludes execution time.

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 3:16 pm

jahboater wrote:
Sat Jun 19, 2021 2:05 pm
If that change is accepted, then signed integer overflow will stop being UB from C++20
(Because with only one representation supported, there is only one possible result for overflow, therefore it becomes "defined").
That is good news. A standard should reflect common practice.
jahboater wrote:
Sat Jun 19, 2021 2:05 pm
If the magic switch simply makes it go faster then I would argue that the behavior is the same.
Similar to how the "as if" rule excludes execution time.
If the observable behaviour is the same with and without the magic switch, apart from a performance boost due to optimisation, then I say all is well.

But then my question is: If the observable behaviour is the same then why is there a need for the magic switch? Why not just do that all the time at -O3 or whatever? Like all other standard compliant optimisations that follow the "as if" rule.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 4:09 pm

Heater wrote:
Sat Jun 19, 2021 3:16 pm
But then my question is: If the observable behaviour is the same then why is there a need for the magic switch? Why not just do that all the time at -O3 or whatever? Like all other standard compliant optimisations that follow the "as if" rule.
I think the compiler always has to cover all possibilities, deal correctly with worst cases, and be strictly standards compliant.

For the -ffinite-math-only option, that's the user saying to the compiler "my code only deals with finite numbers", so the compiler produces faster code for the users program. But no compiler could ever assume that by default! So we must have a switch the user can invoke. The resulting code produces identical results, but it no longer has all the unneeded checks.

There are many many similar options:-

-fomit-frame-pointer and -fno-unwind-tables // no good if you plan to use a debugger

-fno-math-errno // the user relies on IEEE exceptions and doesn't use errno. This means for example that the compiler could emit the fsqrt instruction which doesn't set errno but is very fast, instead of a library routine.

-fstrict-aliasing // the compiler must assume pointer aliasing and produce correct but slow code. You can tell the compiler that your particular program follows all the aliasing rules, allowing faster code to be emitted.

There are plenty of others.
In all these cases, the results are the same (apart from speed/code size).
The compiler cannot make assumptions, it must always produce rigorously correct standards compliant code in all circumstances.
But the user can tell it otherwise - then its the users fault if the code breaks!

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 4:32 pm

jahboater wrote:
Sat Jun 19, 2021 4:09 pm
I think the compiler always has to cover all possibilities, deal correctly with worst cases, and be strictly standards compliant.
Good. I hope so too.
jahboater wrote:
Sat Jun 19, 2021 4:09 pm
For the -ffinite-math-only option, that's the user saying to the compiler "my code only deals with finite numbers", so the compiler produces faster code for the users program. But no compiler could ever assume that by default! So we must have a switch the user can invoke. The resulting code produces identical results, but it no longer has all the unneeded checks.
I find that confusing.

If whatever magic compiler switch changes the observable behaviour of whatever data type then a reader of the source cannot tell what happens simply from the source he is reading.

That is the thing that niggles me.

Effectively that magic switch has changed the behaviour of the data type. That is to say some type T is now some type T', which behaves subtly differently. From reading the source code one can no longer tell how T behaves, not without knowing what compiler and what magic switch is used.

It would be better if the language had different types, T and T' and whatever that spelled out those different behaviours for the reader of the source.
jahboater wrote:
Sat Jun 19, 2021 4:09 pm
There are many many similar options:-
No doubt.
jahboater wrote:
Sat Jun 19, 2021 4:09 pm
But the user can tell it otherwise - then its the users fault if the code breaks!
I presume when you say user you mean the programmer using the compiler.

So, ah yes, if the compiler has switches to stop it enforcing the C standard then we are not talking about the C language anymore.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 4:57 pm

Heater wrote:
Sat Jun 19, 2021 4:32 pm
If whatever magic compiler switch changes the observable behaviour of whatever data type then a reader of the source cannot tell what happens simply from the source he is reading.

That is the thing that niggles me.
Apart from the execution, time the observable behaviour does not change.
So no need to be niggled :)

How does telling the compiler that your code only deals with finite values change the result?
It only means the compiler and the math library don't have to waste time checking for NaN's or infinity that will never occur.

For interest, the math library contains a parallel set of functions that work with known to be finite arguments.
They have names like: __pow_finite() and __acos_finite() etc.

I am certain they produce the same results.

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 5:20 pm

jahboater wrote:
Sat Jun 19, 2021 4:57 pm
Apart from the execution, time the observable behaviour does not change.
So no need to be niggled :)
But, but, if the observable behaviour of whatever functions source code I'm looking at does not change why is there a need for the switch? Just optimise things like everything else when optimisations are turned on.
jahboater wrote:
Sat Jun 19, 2021 4:57 pm
How does telling the compiler that your code only deals with finite values change the result?
It only means the compiler and the math library don't have to waste time checking for NaN's or infinity that will never occur.
How do I know that the code only deals with finite values?

What I mean is, when I look at the source of a 10 line function in a 100,000 line code base, it would be nice to know exactly what that code will do. Without having to understand the other 100,000 - 10 lines of code in the entire program.

As far as I can tell, making such behaviour clear from the source requires either:
a) Different data types. Say "float" and "fast_float" that deal with NaNs or whatever differently.
b) Different operators for arithmetic operations that apply different rules to the same type.

The C/C++ language has neither. Instead we have to know how the code was built to know what will happen.

Perhaps that is not a big deal for you. It niggles me.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 7:07 pm

Heater wrote:
Sat Jun 19, 2021 5:20 pm
Perhaps that is not a big deal for you. It niggles me.
It doesn't niggle me because:

Left to itself the compiler will produce correct code that complies with IEEE 754 (so be consistent and reproducible) and will be as accurate as possible.

I believe this is true for Fortran and C with their vast user bases, long history, slow and careful standards development etc etc.

All through the compiler and math library documentation, its very clear that standards compliance first, then accuracy are the prime directives, with the user (developer) having to take explicit and deliberate action to move away from that.

I believe that my level of understanding of floating-point concepts, the environment, the math library is the limiting factor!
The C language, C compiler, math library, IEEE standards are produced by people with far greater depth of knowledge and experience,
so I have to trust them. Furthermore, all this stuff is so mature and has such a wide exposure that significant bugs will have been found long ago.

So no, I don't worry, and if I did find an issue, further study should resolve it.

(BTW I would not be happy with a language that invented its own FP format (e.g. XML as noted above), but that's another story).

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 8:02 pm

jahboater wrote:
Sat Jun 19, 2021 7:07 pm
It doesn't niggle me because:
It's starting to niggle me even more because after ten mins of goggling around I can't find a strict definition of what "-ffinite-math-only" actually does.
jahboater wrote:
Sat Jun 19, 2021 7:07 pm
Left to itself the compiler will produce correct code that complies with IEEE 754 (so be consistent and reproducible) and will be as accurate as possible.
What I did find was this: https://gnu.huihoo.org/gcc/gcc-3.4.6/g7 ... tions.html which says:
-ffinite-math-only
Allow optimizations for floating-point arithmetic that assume that arguments and results are not NaNs or +-Infs.
This option should never be turned on by any -O option since it can result in incorrect output for programs which depend on an exact implementation of IEEE or ISO rules/specifications.

The default is -fno-finite-math-only.
Which basically tells me the exact opposite of what you have been saying. That setting the switch will change behaviour and break IEEEE or ISO standards compliance.

Colour me confused.
Memory in C++ is a leaky abstraction .

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 8:10 pm

jahboater wrote:
Sat Jun 19, 2021 7:07 pm
(BTW I would not be happy with a language that invented its own FP format (e.g. XML as noted above), but that's another story).
I agree. We have languages that have types like float, double, f32, f64 and whatever else. Mostly they follow IEEE 754 now a days.

With my hint that a language might have other floating point types I would assume they have different names for them. For example 16 bit floats are now a widely used thing in the neural network world (Is there an IEEE standard for that?). So why not a float16_t or f16 type for example?

My claim there was that floats that behave differently in the face of NaNs or whatever are already different types. That those types should have different names in the language, rather than being switched in and out by some obscure compiler option.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 8:16 pm

Heater wrote:
Sat Jun 19, 2021 8:02 pm
-ffinite-math-only
Allow optimizations for floating-point arithmetic that assume that arguments and results are not NaNs or +-Infs.
This option should never be turned on by any -O option since it can result in incorrect output for programs which depend on an exact implementation of IEEE or ISO rules/specifications.

The default is -fno-finite-math-only.
Which basically tells me the exact opposite of what you have been saying. That setting the switch will change behaviour and break IEEEE or ISO standards compliance.
No it does not. Iff your code incorrectly produced values that were NaN or infinity, then yes indeed it might break the standards compliance (not sure exactly what it would actually do, it seems a daft thing to try).

However we (the developers) know for certain that our code only produces finite values, and there is not a problem. If we were not sure then we would not use this option, or would implement some kind of checks to validate input and results.

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 8:26 pm

jahboater wrote:
Sat Jun 19, 2021 8:16 pm
No it does not. Iff your code incorrectly produced values that were NaN or infinity, then yes indeed it might break the standards compliance (not sure exactly what it would actually do, it seems a daft thing to try).
OK.

Because I'm daft I have just been trying to craft an example where using "-ffinite-math-only" did cause a program to behave differently :)

No luck so far. Do you have such an example?
jahboater wrote:
Sat Jun 19, 2021 8:16 pm
However we (the developers) know for certain that our code only produces finite values, and there is not a problem. If we were not sure then we would not use this option, or would implement some kind of checks to validate input and results.
I presume you mean "Iff we (the developers) know for certain...." As in "if and only if".

Granted that is possible for trivial code.

I'm not sure its possible in general. In the same way developers cannot be sure they don't overflow integer ranges, array bounds, and so on. As the history of bugs in almost everything demonstrates.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 8:38 pm

Heater wrote:
Sat Jun 19, 2021 8:26 pm
Because I'm daft I have just been trying to craft an example where using "-ffinite-math-only" did cause a program to behave differently :)
If you succeed with finite values, then perhaps raise a bug. You would expect the handling of NaN's etc to be different of course, that's the point of it.
If you succeed in finding a difference when using finite values, I'll stop using the option immediately. So I am interested in your results !!
Heater wrote:
Sat Jun 19, 2021 8:26 pm
No luck so far. Do you have such an example?
No. I have been using it for years, but never did any really comprehensive tests, I just trust the compiler devs :(
Heater wrote:
Sat Jun 19, 2021 8:26 pm
I presume you mean "Iff we (the developers) know for certain...." As in "if and only if".
Yes. I knew you would know what it meant :) Perhaps should be IFF in upper case.
Heater wrote:
Sat Jun 19, 2021 8:26 pm
Granted that is possible for trivial code.

I'm not sure its possible in general. In the same way developers cannot be sure they don't overflow integer ranges, array bounds, and so on. As the history of bugs in almost everything demonstrates.
Exactly. Agreed. I would not use this option unless I could prove that finite only values were being used.
I don't know how _acos_finite() handles a non-finite argument and I don't really want to find out!

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 8:58 pm

jahboater wrote:
Sat Jun 19, 2021 8:38 pm
If you succeed with finite values, then perhaps raise a bug. You would expect the handling of NaN's etc to be different of course, that's the point of it.
If you succeed in finding a difference when using finite values, I'll stop using the option immediately. So I am interested in your results !!
Given the almost non-existent documentation on what "-ffinite-math-only" is actually supposed to do (or allow) it's kind of difficult to think what an example that shows a difference would look like. Except we know it's something to do with ignoring NaNs and Infs etc.

So of course my attempt at showing a difference are crafted to create and test for NaNs and Infs. For example this little effort:

Code: Select all

#include <stdio.h>
#include <math.h>

int main () {
    double d = 1.0;
    int n = 1;

    while (1) {
	d = d * (d + n);
	n = n + 1;
	if (isinf(d)) {
            printf("Reached infinity!\n");
	    break;
	} else {
	    printf("%f\n", d);
	}
    }
}
But around here that produces the same output with and without "-ffinite-math-only":

Code: Select all

2.000000
8.000000
88.000000
8096.000000
65585696.000000
4301483913318592.000000
18502763856538658091431257702400.000000
342352270330833316553931419600494275580842436708707000983224320.000000
117205077000675979660092616418905375858663841351211895762202291450908654010073928823543639394442223267343126475248829284220928.000000
13737030074734386018690567769506813859265633337121971160217381706408245079902446302289864372677766231881374466874680411677685946994613518250436035965467569431849489098855134053023629634048332869293547671225245759904564595993318581339974799652114399232.000000
Reached infinity!  
Will try some more....
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 10:17 pm

It does produce different code. This is for the Pi4 in 64-bit mode with GCC 11.1.
As you can see there is quite a lot more code without the option. This is very interesting, its years since I last looked at it.
In the "without option" code it refers to isinf() (which has been elided from the finite only code for obvious reasons), but where is it?
Edit: it loaded the infinity value from static memory and just did a simple compare on line 41.
With -ffinite-math-only

Code: Select all

   1                            .arch armv8-a+crc
   2                            .file   "try.c"
   3                    // GNU C11 (GCC) version 11.1.0 (aarch64-unknown-linux-gnu)
   4                    //      compiled by GNU C version 11.1.0, GMP version 6.1.0, MPFR version 3.
   5
   6                    // GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=13107
   7                    // options passed: -march=armv8-a+crc -mcpu=cortex-a72 -mtune=cortex-a72 -ma
   8                            .text
   9                            .section        .rodata.str1.1,"aMS",@progbits,1
  10                    .LC0:
  11 0000 25660A00              .string "%f\n"
  12                            .section        .text.startup,"ax",@progbits
  13                            .align  2
  14                            .global main
  15                            .hidden main
  16                            .type   main, %function
  17                    main:
  18 0000 F37BBEA9              stp     x19, x30, [sp, -32]!    //,,,
  19                    // try.c:6:     int n = 1;
  20 0004 33008052              mov     w19, 1  // n,
  21                    // try.c:4: int main () {
  22 0008 E80B00FD              str     d8, [sp, 16]    //,
  23                    // try.c:5:     double d = 1.0;
  24 000c 08106E1E              fmov    d8, 1.0e+0      // d,
  25                    .L2:
  26                    // try.c:9:   d = d * (d + n);
  27 0010 6002621E              scvtf   d0, w19 // tmp97, n
  28                    // try.c:15:       printf("%f\n", d);
  29 0014 00000010              adr     x0, .LC0        //,
  30                    // try.c:10:   n = n + 1;
  31 0018 73060011              add     w19, w19, 1     // n, n,
  32                    // try.c:9:   d = d * (d + n);
  33 001c 0028681E              fadd    d0, d0, d8      // _2, tmp97, d
  34                    // try.c:9:   d = d * (d + n);
  35 0020 0809601E              fmul    d8, d8, d0      // d, d, _2
  36                    // try.c:15:       printf("%f\n", d);
  37 0024 0041601E              fmov    d0, d8  //, d
  38 0028 00000094              bl      printf          //
  39 002c F9FFFF17              b       .L2             //
  40                            .size   main, .-main
  41                            .ident  "GCC: (GNU) 11.1.0"
and without:

Code: Select all

   1                            .arch armv8-a+crc
   2                            .file   "try.c"
   3                    // GNU C11 (GCC) version 11.1.0 (aarch64-unknown-linux-gnu)
   4                    //      compiled by GNU C version 11.1.0, GMP version 6.1.0, MPFR version 3.
   5
   6                    // GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=13107
   7                    // options passed: -march=armv8-a+crc -mcpu=cortex-a72 -mtune=cortex-a72 -ma
   8                            .text
   9                            .section        .rodata.str1.1,"aMS",@progbits,1
  10                    .LC0:
  11 0000 52656163              .string "Reached infinity!"
  11      68656420
  11      696E6669
  11      6E697479
  11      2100
  12                    .LC1:
  13 0012 25660A00              .string "%f\n"
  14                            .section        .text.startup,"ax",@progbits
  15                            .align  2
  16                            .global main
  17                            .hidden main
  18                            .type   main, %function
  19                    main:
  20 0000 F37BBEA9              stp     x19, x30, [sp, -32]!    //,,,
  21                    // try.c:6:     int n = 1;
  22 0004 33008052              mov     w19, 1  // n,
  23                    // try.c:4: int main () {
  24 0008 E827016D              stp     d8, d9, [sp, 16]        //,,
  25                    // try.c:5:     double d = 1.0;
  26 000c 08106E1E              fmov    d8, 1.0e+0      // d,
  27                    // try.c:11:   if (isinf(d)) {
  28 0010 4902005C              ldr     d9, .LC2        // tmp100,
  29                    .L3:
  30                    // try.c:9:   d = d * (d + n);
  31 0014 6002621E              scvtf   d0, w19 // tmp98, n
  32                    // try.c:10:   n = n + 1;
  33 0018 73060011              add     w19, w19, 1     // n, n,
  34                    // try.c:9:   d = d * (d + n);
  35 001c 0028681E              fadd    d0, d0, d8      // _2, tmp98, d
  36                    // try.c:9:   d = d * (d + n);
  37 0020 0809601E              fmul    d8, d8, d0      // d, d, _2
  38                    // try.c:11:   if (isinf(d)) {
  39 0024 00C1601E              fabs    d0, d8  // tmp99, d
  40                    // try.c:11:   if (isinf(d)) {
  41 0028 0020691E              fcmp    d0, d9  // tmp99, tmp100
  42 002c ED000054              ble     .L2             //,
  43                    // try.c:12:        printf("Reached infinity!\n");
  44 0030 00000010              adr     x0, .LC0        //,
  45 0034 00000094              bl      puts            //
  46                    // try.c:18: }
  47 0038 00008052              mov     w0, 0   //,
  48 003c E827416D              ldp     d8, d9, [sp, 16]        //,,
  49 0040 F37BC2A8              ldp     x19, x30, [sp], 32      //,,,
  50 0044 C0035FD6              ret
  51                    .L2:
  52                    // try.c:15:       printf("%f\n", d);
  53 0048 0041601E              fmov    d0, d8  //, d
  54 004c 00000010              adr     x0, .LC1        //,
  55 0050 00000094              bl      printf          //
  56                    // try.c:9:   d = d * (d + n);
  57 0054 F0FFFF17              b       .L3             //
  58                            .size   main, .-main
  59                            .align  3
  60                    .LC2:
  61 0058 FFFFFFFF              .word   -1
  62 005c FFFFEF7F              .word   2146435071
  63                            .ident  "GCC: (GNU) 11.1.0"
Last edited by jahboater on Sat Jun 19, 2021 10:35 pm, edited 1 time in total.

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 10:25 pm

The results are different on my Pi.
Without the -ffinite-math-only option it behaves as your code does.
With the -ffinite-math-only option it fails to spot the infinity condition because the check has been removed, and loops forever.
That's as expected I suppose.

Code: Select all

$ ./try
2.000000
8.000000
88.000000
8096.000000
65585696.000000
4301483913318592.000000
18502763856538658091431257702400.000000
342352270330833316553931419600494275580842436708707000983224320.000000
117205077000675979660092616418905375858663841351211895762202291450908654010073928823543639394442223267343126475248829284220928.000000
13737030074734386018690567769506813859265633337121971160217381706408245079902446302289864372677766231881374466874680411677685946994613518250436035965467569431849489098855134053023629634048332869293547671225245759904564595993318581339974799652114399232.000000
inf
inf
inf
inf
inf
inf
inf
inf
.... infinite loop
What I would like to see (or better not to see:) ) is different results with finite only values.
Last edited by jahboater on Sat Jun 19, 2021 10:37 pm, edited 1 time in total.

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sat Jun 19, 2021 10:31 pm

Years ago, I saw that the compiler was producing checks for NaN's before each and every "x > y" sort of floating-point comparison.
Proving that it was "ordered".
It doesn't seem to do that now, I don't know why.
-ffinite-math-only removed those checks of course.

lurk101
Posts: 689
Joined: Mon Jan 27, 2020 2:35 pm
Location: Cumming, GA (US)

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 2:43 am

Heater wrote:
Sat Jun 19, 2021 3:16 pm
But then my question is: If the observable behaviour is the same then why is there a need for the magic switch? Why not just do that all the time at -O3 or whatever?
Use -O3 all the time? Pretty much what I do! Good question.

I've compiled kernels at -O3. They ran fine.
How to make your arguments stronger? Longer is not the answer.

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 5:24 am

lurk101 wrote:
Sun Jun 20, 2021 2:43 am
Heater wrote:
Sat Jun 19, 2021 3:16 pm
But then my question is: If the observable behaviour is the same then why is there a need for the magic switch? Why not just do that all the time at -O3 or whatever?
Use -O3 all the time? Pretty much what I do! Good question.

I've compiled kernels at -O3. They ran fine.
I was not concerned with the "-O3". Optimisations do not break standards compliant code.

It's the "-ffinite-math-only", which apparently does break standards compliant code. Oddly producing different behaviour with gcc-10 and clang 12.0.5 on my Mac Book, where it always detects infinity, vs gcc 7.5 on a Jetson NX, where it fails to detect infinity.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 6:07 am

Heater wrote:
Sun Jun 20, 2021 5:24 am
It's the "-ffinite-math-only", which apparently does break standards compliant code. Oddly producing different behaviour with gcc-10 and clang 12.0.5 on my Mac Book, where it always detects infinity, vs gcc 7.5 on a Jetson NX, where it fails to detect infinity.
By using "-ffinite-math-only", you declared to the compiler that your program never has any infinity or NaN data values.

Why are you interested in what happens when your program incorrectly has an infinite value?
What do you expect it to do?

It could fail the compilation, but the isinf() is still legal C.
Instead, knowing that isinf() cannot possibly ever return true (because the user said so), the optimizer deletes the test and the dependent code.
Heater wrote:
Sun Jun 20, 2021 5:24 am
It's the "-ffinite-math-only", which apparently does break standards compliant code.
When used correctly, it does not (as far as I can see).
Unless you can show me an example of C code with finite data values that produces non-compliant results with this option.

This is a bit like the "no-aliasing" declaration (-fstrict-aliasing for GCC) that compilers have had since the dawn of time.
If you pass -fstrict-aliasing to the compiler, the optimizer will take advantage of it.
Everyone knows that if you did then have some pointer aliasing in your program it would probably fail.
No surprise, no big deal.

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 8:55 am

I see that -ffast-math turns on -ffinite-math-only, which I find surprising as this disallows non-finite values.
-ffast-math
Sets the options -fno-math-errno, -funsafe-math-optimizations, -ffinite-math-only,
-fno-rounding-math, -fno-signaling-nans, -fcx-limited-range and
-fexcess-precision=fast.

This option causes the preprocessor macro "__FAST_MATH__" to be defined.

This option is not turned on by any -O option besides -Ofast since it can result in
incorrect output for programs that depend on an exact implementation of IEEE or ISO
rules/specifications for math functions. It may, however, yield faster code for
programs that do not require the guarantees of these specifications.
The manual page entry for -ffinite-math-only says:
-ffinite-math-only
Allow optimizations for floating-point arithmetic that assume that arguments and
results are not NaNs or +-Infs.

This option is not turned on by any -O option since it can result in incorrect output
for programs that depend on an exact implementation of IEEE or ISO
rules/specifications for math functions
. It may, however, yield faster code for
programs that do not require the guarantees of these specifications.

The default is -fno-finite-math-only.
again, I hope the math functions behave correctly when given finite arguments.
But it doesn't actually say so ....
I see (at least) these math library functions have versions for finite only arguments:

Code: Select all

__pow_finite
__log_finite
__log2_finite
__log10_finite
__exp_finite
__exp2_finite
__exp10_finite
__asin_finite
__acos_finite
which might be worth investigating.

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 9:54 am

jahboater wrote:
Sun Jun 20, 2021 6:07 am
Why are you interested in what happens when your program incorrectly has an infinite value?
What do you expect it to do?
There are a few things I would like to see in a high-level language. They are all related but somewhat orthogonal to each other:

a) Reproducibility. That the same source code should produce the same results no matter which compiler is used and no matter what hardware. That way we can be sure no random bugs appear when moving code around. People consider reproducibility in science to be of fundamental importance. Reproducibility in maths is crucial. How did it happen that us software engineers talked ourselves into ignoring reproducibility?

The reproducibility I have in mind is reproducing the behaviours as seen in the source code alone. I'm really with Tony Hoare here:
The behavior of every syntactically correct program should be completely predictable from its source code. For the sake of safety, security, and programmer sanity, it must be impossible for a program to "run wild."
From the source code. Not from a deep knowledge of external things like compiler flags, implementation define features, etc, etc.

b) Correctness. As you say "what do you expect it to do?".

Good question. There is some wiggle room here. As minimum it should always do the same thing, as above. Details are up for debate. Taking a simpler example: I have always thought integer types should immediately halt the program on overflow with a suitable error message. That would satisfy Hoare's philosophy. Ah, you say, what if I want wrapping semantics because that is very useful in many cases. Or what about saturating semantics? No probs. In a high level language they would be different integer types, use the type you want.

I probably have never used floating point types enough to really think about what I really want them to do in detail.
jahboater wrote:
Sun Jun 20, 2021 6:07 am
When used correctly, it does not (as far as I can see).
Unless you can show me an example of C code with finite data values that produces non-compliant results with this option.
I'm willing to believe that.

Still, we have a simple example code here that behaves differently with different machines and compilers. I consider that as bad as having some kind of mathematical proof that causes different mathematicians to come to different conclusions. We would not like that would we?
jahboater wrote:
Sun Jun 20, 2021 6:07 am
This is a bit like the "no-aliasing" declaration (-fstrict-aliasing for GCC) that compilers have had since the dawn of time.
If you pass -fstrict-aliasing to the compiler, the optimizer will take advantage of it.
Everyone knows that if you did then have some pointer aliasing in your program it would probably fail.
No surprise, no big deal.
That is another can of worms...
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 10:00 am

Heater,

You might find these GCC builtins useful for testing:

Code: Select all

double x;
x = __builtin_inf();       // Positive Infinity  (or use the C99 standard macro INFINITY)
x = __builtin_nan("0");    // Quiet NaN - tag bits zero 
x = __builtin_nans("0");   // Signaling NaN - tag bits zero 
They should work for Clang too.

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 4:51 pm

Heater wrote:
Sun Jun 20, 2021 9:54 am
Still, we have a simple example code here that behaves differently with different machines and compilers. I consider that as bad as having some kind of mathematical proof that causes different mathematicians to come to different conclusions. We would not like that would we?
Compilers don't call things "undefined behavior", that's the domain of the language standards.
However what you have done is tantamount to undefined behavior.

You have informed the compiler that your C code has finite data values only.
Then you have used infinity!

What do expect to happen?
Did you think the compiler would predict somehow that "d" would reach infinity and cancel the -ffinite-math-only option?
Or did you expect the compiler to simply ignore the -ffinite-math-only option and not do any related optimizations.
By the sound of it, you would prefer the latter.

Just like UB I'm not at all surprised it behaves differently with different compilers.
It seems perfectly reasonably to me.

There are many compiler options where you declare to the compiler that xxx is true for your specific piece of code.
These options are provided so you can help the optimizer produce faster code than would be possible if it had to assume the worst case all the time.

But then its your responsibility to ensure the code complies with your declaration(s).
Woe betide you if you lie to the compiler.
Its not the compilers fault if things break when you lie to it!
You certainly cannot reasonably expect reproducibility and correctness when you lie to it :(

I think of it as a "contract" with the compiler.

Heater
Posts: 18362
Joined: Tue Jul 17, 2012 3:02 pm

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 7:24 pm

jahboater wrote:
Sun Jun 20, 2021 4:51 pm
Compilers don't call things "undefined behavior", that's the domain of the language standards.
Agreed.
jahboater wrote:
Sun Jun 20, 2021 4:51 pm
However what you have done is tantamount to undefined behavior.

You have informed the compiler that your C code has finite data values only.
Then you have used infinity!
Agreed.

Actually, to cut this short, I have to say I agree with everything you have said there. This is C/C++ we are talking about. Having used C since the early 80's and C++ since the late 90's, having spend countless hours debugging code, create by myself and others, I get the idea.
jahboater wrote:
Sun Jun 20, 2021 4:51 pm
I think of it as a "contract" with the compiler.
However, here we may differ.

I don't want to make a contract with a particular compiler implementation or hardware architecture. I want to make a contract with a high level programming language definition. I want any source code I have to compile to something that behaves exactly the same, everywhere. As specified in the source code and the language definition.

In short, I want a high level language, like C/C++, that does not leave so many things to "Undefined Behaviour" or "Implementation Defined" and the such woollieness.

The sad truth is that we "lie to our compilers" all the time. By mistake. Things go wrong. These are called "bugs". Much time and expense goes into tracking down and fixing those bugs. Many security vulnerabilities are a result of such bugs.

Since the four decades or more of programming language design and compiler construction research my dreams are now possible (To a large extent).

I find that a very attractive proposition. Something we have not had on offer since ALGOL.

I can understand if it is of no concern to you.

Whilst we are here, how come when I do:

Code: Select all

printf("%f\n", d);
I get ludicrous output like:

Code: Select all

89884656743115795386465259539451236680898848947115328636715040578866337902750481566354238661203768010560056939935696678829394884407208311246423715319737062188883946712432742638151109800623047059726541476042502884419075341171231440736956555270413618581675255342293149119973622969239858152417678164812112068608.000000
Where did all those significant digits come from? Even a double can not contain that.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7194
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: TIL: "==" is not "is equal" for "double"

Sun Jun 20, 2021 7:43 pm

Heater wrote:
Sun Jun 20, 2021 7:24 pm
Whilst we are here, how come when I do:

Code: Select all

printf("%f\n", d);
I get ludicrous output like:

Code: Select all

89884656743115795386465259539451236680898848947115328636715040578866337902750481566354238661203768010560056939935696678829394884407208311246423715319737062188883946712432742638151109800623047059726541476042502884419075341171231440736956555270413618581675255342293149119973622969239858152417678164812112068608.000000
Where did all those significant digits come from? Even a double can not contain that.
Try %g instead, or perhaps %e for the strictly scientific notation.

%f prints the number as ddd.ddd and the given precision says how may digits after the decimal point (defaulting to 6).
It has to print everything before the decimal point in full, even if the available precision has nowhere near enough precision and most of the trailing digits are garbage.

Code: Select all

e, E   The double argument is rounded and converted in the style [-]d.ddde+-dd where there
       is one digit before the decimal-point character and the number of digits  after  it
       is  equal  to  the precision; if the precision is missing, it is taken as 6; if the
       precision is zero, no decimal-point character appears.  An E  conversion  uses  the
       letter  E  (rather than e) to introduce the exponent.  The exponent always contains
       at least two digits; if the value is zero, the exponent is 00.

f, F   The double argument is rounded and converted  to  decimal  notation  in  the  style
       [-]ddd.ddd,  where  the number of digits after the decimal-point character is equal
       to the precision specification.  If the precision is missing, it is taken as 6;  if
       the precision is explicitly zero, no decimal-point character appears.  If a decimal
       point appears, at least one digit appears before it.

       (SUSv2 does not know about F and says that  character  string  representations  for
       infinity and NaN may be made available.  SUSv3 adds a specification for F.  The C99
       standard specifies "[-]inf" or "[-]infinity" for infinity, and  a  string  starting
       with  "nan"  for NaN, in the case of f conversion, and "[-]INF" or "[-]INFINITY" or
       "NAN*" in the case of F conversion.)

g, G   The double argument is converted in style f or e (or F or  E  for  G  conversions).
       The  precision  specifies  the  number  of significant digits.  If the precision is
       missing, 6 digits are given; if the precision is zero, it is treated as 1.  Style e
       is  used  if  the  exponent  from its conversion is less than -4 or greater than or
       equal to the precision.  Trailing zeros are removed from the fractional part of the
       result; a decimal point appears only if it is followed by at least one digit.
Last edited by jahboater on Sun Jun 20, 2021 8:01 pm, edited 2 times in total.

Return to “C/C++”