But I can't seem to figure out what kind of compiler logic would give me the above results. Surely all the above variables must be initialized somehow? I understand "undefined behaviour" means exactly that, but I'm straining my brain to find some other way the above code could be interpreted, and can't think of any.
I've tried this on g++ 3.3.6, 3.4.2, and 4.0.2. Only 4.0.2 behaves this way.
Secondly, what is the proper, alias-correct, way to apply structure to a block of data? For example:
Chris Frey wrote: > But I can't seem to figure out what kind of compiler logic would give > me the above results. Surely all the above variables must be initialized > somehow? I understand "undefined behaviour" means exactly that, but I'm > straining my brain to find some other way the above code could be > interpreted, and can't think of any.
Undefined behaviour is not about interpretation of code, but about the results. It means exactly that there is no logic (nor there should be) in how the program will _behave_. In your example (with optimizations turned on) the compiler probably tosses some assembler instructions around to speed up execution of the code, which may in the end yield absolutely _any_ result depending on surrounding code, target architecture etc.
>I've tried this on g++ 3.3.6, 3.4.2, and 4.0.2. Only 4.0.2 behaves this
way.
If you want GCC 4.x to behave as 3.x in this aspect, you can add '-fno-strict-aliasing' to your compile flags.
If you deliberately tell lies to the compiler (like telling it that the address of a double is the address of an uint64_t and then dereferencing that, then all bets are off. It seems that you expect the compiler to conserve bit patterns, why should it? If you dereference a pointer that is of the wrong type for the addressed memory the compiler is allowed to do anything it likes.
> If you deliberately tell lies to the compiler (like telling it that the > address of a double is the address of an uint64_t and then dereferencing > that, then all bets are off. It seems that you expect the compiler to > conserve bit patterns, why should it? If you dereference a pointer that > is of the wrong type for the addressed memory the compiler is allowed to > do anything it likes.
More precisely, the type of value retrieved from the variable disagrees with the type that was stored at that location. Consider a slightly different program:
Even though this program performs the same types of casts as the original, it avoids running into the same problem. Since the value of conv is both stored and retrieved as a double, the behavior here is defined (assuming that a unint64 is of sufficient size and has alignment requirements at least as strict as a double).
The actual problem in the original program is not due to altered bit patterns, but due to unsaved values. In the original example, the optimizer no doubt notes that there is a uint64 value stored in conv, but no subsequent operation exists to retrieve a unit64 value from that location. As consequence, there is no reason to store anything in conv and the statement that does so can be eliminated. With no value being stored in conv, the double retrieved from that location in a subsequent operation is undefined.
Using a union would solve this problem with a gcc compiler:
The Standard states the result of accessing one member of a union after storing a different member are unspecified. gcc for its part specifies that the result in such a case works as expected. Alternately, converting to and from a char * pointer would also avoid the original problem.
> But I can't seem to figure out what kind of compiler logic would give > me the above results. Surely all the above variables must be initialized > somehow?
I don't think so. Since the initializations of both conv and d2 exhibit undefined behavior, an optimizer might choose to do nothing at all...that is, leave conv and d2 uninitialized. That fulfills the requirements of the language (of which there are none, since the behavior is undefined) and is a pretty efficient optimization (zero instructions!).
Of course, g++ might have something cleverer in mind...you could look at the assembly output if you care about what it's actually doing. But the point is that it's not required to do anything at all.
> Even though this program performs the same types of casts as the > original, it avoids running into the same problem. Since the value of > conv is both stored and retrieved as a double, the behavior here is > defined (assuming that a unint64 is of sufficient size and has > alignment requirements at least as strict as a double).
Is it? Because it's not in C (see C99:6.5p6-7). When looking at the corresponding paragraph in the C++ standard (3.10p15), it comes down to the question of what the dynamic type of the object 'conv' is in the third line of main() above. I'd say it's clearly uint64_t, and then accessing it as a double is UB.
Furthermore, the result of the pointer conversion is unspecified (except that converting it back to the original type is guaranteed to yield the original pointer value) (5.2.10p7). Which means that the double * does not necessairly point at the same memory location.
Jan Dvorak wrote: >>I've tried this on g++ 3.3.6, 3.4.2, and 4.0.2. Only 4.0.2 behaves >>this way. > If you want GCC 4.x to behave as 3.x in this aspect, you can add > '-fno-strict-aliasing' to your compile flags.
That sounds like an intelligent solution on the part of g++. Presumably, they would have had the presense of a reinterpret_cast turn on this option automatically, had that been technically feasable.
-- James Kanze mailto: james.ka...@free.fr Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 pl. Pierre Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34
Francis Glassborow wrote: > In article <43912...@news.sentex.net>, Chris Frey <cdf...@sentex.ca> > writes >>I'm hoping someone can help explain the compiler's logic to me >>regarding this example: >>#include <iostream> >>#include <stdint.h> >>int main() >>{ >> double d = 42.42; >> uint64_t conv = *((uint64_t*) &d); >> double d2 = *((double*)&conv); >> std::cout << "d = " << d << " d2 = " << d2 << std::endl; >>} >>When I compile it on g++ 4.0.2, using the following flags, I get: >>$ g++ -O2 -Wall -o swap swap.cc >>$ ./swap >>d = 42.42 d2 = 10.3124 > If you deliberately tell lies to the compiler (like telling it that > the address of a double is the address of an uint64_t and then > dereferencing that, then all bets are off.
Don't be silly. He knows it is undefined behavior according to the C++ standard. On the other hand, the casts are reinterpret_cast's; the results of such conversions are implementation defined, of course, but the intent is that it should be "unsurprising to those who know the addressing structure of the underlying machine." The actual language quoted is from a note concerning reinterpret_cast between integers and pointers, but it doesn't seem unreasonable to apply it here. This is the sort of thing that reinterpret_cast is designed for; looking at the bit pattern of one type as if it were the bit pattern of another type.
What's the point of having reinterpret_cast at all if you cannot use it to do these sort of things. Not portably, of course, but if you know the different representations used on your machine.
Note too that the Open Systems requires something similar to work; see the examples concerning the return value of dlsym.
Finally, of course, the question is: how do you do what he is trying to do? The problem occurs frequently in networking protocols, where one has to convert floating point values to a stream of bytes -- the "standard" solution, supposing that the protocol and the host hardware use the same floating point format (typically IEEE), is to read the floating point value as an unsigned integral type of the same size, and shift this to output the bytes in the right order. One knows it isn't portable -- not every machine is IEEE. But one accepts the simplified solution as a compromize.
This technique is called type punning, and it is essential in certain contexts. Traditionnally, in pre-standard C, it was done either like the above, or by means of a union. Standard C said that the union couldn't be used. Since that date, programmers like myself have been saying you have to do something like the above. If neither are supported, how do you type pun?
> It seems that you expect the compiler to conserve bit patterns, why > should it?
Because it conforms to user expectations. reinterpret_cast is designed as an escape clause, exactly for this sort of thing. Obviously, it is implementation specific -- the standard says "undefined behavior" because there is no reasonable definition one could give which could be applied everywhere. But knowing the underlying hardware representations of all of the types involved, you should be able to figure out the actual behavior for any given platform.
> If you dereference a pointer that is of the wrong type for the > addressed memory the compiler is allowed to do anything it likes.
Sure. The compiler is also allowed to reformat your hard disk if you use << without including <ostream>, even if you've included <iostream>. There's a big difference between what a compiler is allowed to do, and what a responsible compiler will actually do. In the case of reinterpret_cast, there is almost no real use for it that doesn't involve undefined behavior. Behavior that we expect, in fact, the implementation to define.
In the end, it's a quality of implemenation issue. In this case, I suppose that one could argue that he should turn the optimization level down. Or use some special compiler flag to tell the compiler that this needs to be supported. Both are sort of reasonable requirements for this sort of code. But... one could also argue that the compiler should automatically switch modes to the one with the special flag on seeing a reinterpret_cast. It is, after all, a red flag that the programmer is doing something "unusual".
My own opinion is that a compiler which doesn't support this with any combination of options is simply not usable, at least not for networking software, or software which needs some form of portable persistence. A compiler that doesn't automatically switch into the appropriate mode when encountering a reinterpret_cast isn't broken, but it isn't as good as it could be, either.
And a compiler vendor who simply rejects such an error report off-hand, because of the standard, is IMHO both misusing the standard and being extremely irresponsible. (Note the "off-hand" -- I can see rejecting it because the user didn't use the right options, and that the internal structure of the compiler makes propagating the information that a reinterpret_cast was present down to the optimizer unfeasable. Saying that "we'd like to do it, but we can't, and here's a work-around" is a bit different than saying "we don't give two shit about your code, as long as we can find an excuse in the standard for not supporting it.")
-- James Kanze mailto: james.ka...@free.fr Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 pl. Pierre Sémard, 78210 St.-Cyr-l'École, France +33 (0)1 30 23 00 34
Niklas Matthies wrote: > On 2005-12-04 13:24, Greg Herlihy wrote: > > Consider a slightly different program: > > int main() > > { > > double d = 42.42; > > uint64_t conv; > > *(double *) &conv = d; > > double d2 = *((double*) &conv); > > std::cout << "d = " << d << " d2 = " << d2 << std::endl; > > } > > Even though this program performs the same types of casts as > > the original, it avoids running into the same problem. Since > > the value of conv is both stored and retrieved as a double, > > the behavior here is defined (assuming that a unint64 is of > > sufficient size and has alignment requirements at least as > > strict as a double). > Is it? Because it's not in C (see C99:6.5p6-7). When looking > at the corresponding paragraph in the C++ standard (3.10p15), > it comes down to the question of what the dynamic type of the > object 'conv' is in the third line of main() above. I'd say > it's clearly uint64_t, and then accessing it as a double is > UB. > Furthermore, the result of the pointer conversion is > unspecified (except that converting it back to the original > type is guaranteed to yield the original pointer value) > (5.2.10p7). Which means that the double * does not necessairly > point at the same memory location.
The standard doesn't define more, because there's not much more that you can say which can be applied portably. It is clear, however, that the intent of reinterpret_cast is to be unsurprising to someone familiar with the architecture of the machine. The whole purpose of reinterpret_cast is to be able to get behind the scenes, so to speak. You don't use it except in low level code, and you don't use it unless you know what you are doing. If you do use it, however, the compiler is supposed to shut up, and do what you tell it to. No questions asked.
In this case, there was nothing fundamentally wrong with the initial code. It makes a certain number of assumptions concerning the underlying hardware, so it is definitly not portable. On the other hand, if those assumptions are met by the hardware, the code should work. As a quality of implementation issue, of course, and according to the intent of the standard -- there's no way to say what it actually is expected to do in standardize, since what it is expected to do depends on the hardware (and may be a program crash or worse on certain hardware). But what is the purpose of providing reinterpret_cast if this sort of thing doesn't work.
-- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
> Niklas Matthies wrote: : >> Is it? Because it's not in C (see C99:6.5p6-7). When looking >> at the corresponding paragraph in the C++ standard (3.10p15), >> it comes down to the question of what the dynamic type of the >> object 'conv' is in the third line of main() above. I'd say >> it's clearly uint64_t, and then accessing it as a double is >> UB.
>> Furthermore, the result of the pointer conversion is >> unspecified (except that converting it back to the original >> type is guaranteed to yield the original pointer value) >> (5.2.10p7). Which means that the double * does not necessairly >> point at the same memory location.
> The standard doesn't define more, because there's not much more > that you can say which can be applied portably.
The standard could very well define that the pointer points to the same memory location (provided the alignment requirements are met as is already required).
:
> In this case, there was nothing fundamentally wrong with the > initial code. It makes a certain number of assumptions > concerning the underlying hardware, so it is definitly not > portable.
The code also makes the more important assumption that the compiler assumes aliasing between lvalues of unrelated types. In my opinion that is fundamentally wrong.
Niklas Matthies wrote: > Furthermore, the result of the pointer conversion is unspecified > (except that converting it back to the original type is guaranteed to > yield the original pointer value) (5.2.10p7). Which means that the > double * does not necessairly point at the same memory location.
Actually it does - this can be inferred from the often-overlooked requirement "The result is an lvalue that _refers to the same object_ as the source lvalue, but with a different type." from § 5.2.10/10. While this paragraph speaks about reference reinterpret_cast as in uint64_t conv; reinterpret_cast<double&>(conv) = 42.42; the guarantee of "referring to the same object" extends to pointers since reference reinterpret_cast and pointer reinterpret_cast are linked as specified in § 5.2.10/10. Of course, in the general case there *IS* a problem when the source lvalue is not properly aligned for the destination type - § 5.2.10/10 doesn't explain what "referring to the same object" is supposed to mean in this case; and § 5.2.10/7 wouldn't apply neither. However, under the condition that uint64_t and double have the same size, I would expect them to have the same alignment requirement.
Nevertheless, I agree that the OP's code can break because of the aliasing rules from § 3.10/15. This paragraph restricts how a conforming program can access the stored value of an object through an lvalue of another type - I imagine that these restrictions exist to give the compiler better opportunities for optimisation. However, this paragraph doesn't restrict storing a new value into an object through an lvalue of another object.
The most portable (for a suitable definition of "portable") way to achieve what the OP wants seems to be std::memcpy(&conv, &d, sizeof conv); best accompanied with something like BOOST_STATIC_ASSERT(sizeof conv == sizeof d);
Niklas Matthies wrote: > On 2005-12-05 12:29, kanze wrote: > > Niklas Matthies wrote: > >> Is it? Because it's not in C (see C99:6.5p6-7). When > >> looking at the corresponding paragraph in the C++ standard > >> (3.10p15), it comes down to the question of what the > >> dynamic type of the object 'conv' is in the third line of > >> main() above. I'd say it's clearly uint64_t, and then > >> accessing it as a double is UB. > >> Furthermore, the result of the pointer conversion is > >> unspecified (except that converting it back to the original > >> type is guaranteed to yield the original pointer value) > >> (5.2.10p7). Which means that the double * does not > >> necessairly point at the same memory location. > > The standard doesn't define more, because there's not much > > more that you can say which can be applied portably. > The standard could very well define that the pointer points to > the same memory location (provided the alignment requirements > are met as is already required).
The standard could say that the resulting pointer points to the same memory location, except when it doesn't. What is the difference between this and undefined behavior?
> > In this case, there was nothing fundamentally wrong with the > > initial code. It makes a certain number of assumptions > > concerning the underlying hardware, so it is definitly not > > portable. > The code also makes the more important assumption that the > compiler assumes aliasing between lvalues of unrelated types. > In my opinion that is fundamentally wrong.
In general I agree. In the specific case in question, however, the aliasing was directly visible -- the compiler knew that the programmer was playing games. (Admitted, it would have been clearer if the programmer had actually written reinterpret_cast. But internally, the compiler should treat the two alike.)
With regards to quality of implementation, we really have two contradictory requirements. On one hand, aliasing has an extremely deleterious effect on optimizing; a compiler should do everything possible to eliminate it, and considering that two pointers to different types cannot be aliases is an important optimization measure, which a good implementation cannot neglect. On the other, in low level code, such type punning is occasionnally necessary. If the goal is that there is to be no need for a lower level language than C++, this sort of thing must be supported.
IMHO, the standard is pretty close to the correct compromize with regards to what a standard can say. It defines the syntax for reinterpret_cast. Just about anything you do with reinterpret_cast -- at least anything useful -- is either undefined or implementation defined behavior. But there are enough suggestions in the standard (and historical precedance) that an "expected" behavior can be deduced for a given architecture. So the only real problem remains the conflict between this expected behavior and the optimization opportunities lost to aliasing. About the only global solution I see (except for compilers which few the entire program and track all possible aliasing) is a compiler option or a pragma -- you tell the compiler that it should consider such aliasing. Ideally, of course, in specific cases where the reinterpret_cast is visible to the compiler, the compiler could invoke the option implicitly.
FWIW: historically, in K&R C (or at least Johnson's pcc), there where three was of doing such type punning:
1. union U { uint32_t raw ; float f ; } tmp ; tmp.f = floatIn ; rawBitsOut = tmp.raw ;
2. uint32_t raw ; (float)raw = floatIn ;
3. uint32_t raw ; *(float&)(&raw) = floatIn ; /* OR */ raw = *(uint32_t)(&floatIn) ;
I think the fact that 2 worked was more or less due to a bug in the compiler; it was never clear that it was supported, and of course, it is definitly banned by the C standard, and all of its successors.
My understanding of the situation is that the C standard also condemns 1. Not just formally, but with the intent that it shouldn't be expected to be supported. It's hard to be 100%, of course, because we are talking about intent, and what one should reasonably be able to expect from a quality implementation, and not about strict conformance.
That leaves 3. If we accept the intention that there should be no place for a lower level language than C++, and that there is no intent to support 1, then we have to suppose that there is an intent to support 3 (via reinterpret_cast in C++). The argument is strengthened by the fact that Posix requires support for 3 in certain specific cases (getting a pointer to a function from the return value of dlsym).
Finally, I might mention that in theory, any such code is totally unportable. So there should be no objection to a pragma. In practice, however, we must recognize the fact that most architectures today are very similar: when was the last time you used a machine which wasn't byte addressed, and whose word size, expressed in bytes, wasn't a power of two? So pragmatically, it would be nice to have a "portable" pragma for this; a portable means of telling the compiler that it must take aliasing between pointers of different types into account. I don't quite see where such a thing fits into the standard, however.
-- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Falk Tannhäuser wrote: > Niklas Matthies wrote: > > Furthermore, the result of the pointer conversion is > > unspecified (except that converting it back to the original > > type is guaranteed to yield the original pointer value) > > (5.2.10p7). Which means that the double * does not > > necessairly point at the same memory location. > Actually it does - this can be inferred from the > often-overlooked requirement "The result is an lvalue that > _refers to the same object_ as the source lvalue, but with a > different type." from § 5.2.10/10. While this paragraph speaks > about reference reinterpret_cast as in > uint64_t conv; > reinterpret_cast<double&>(conv) = 42.42; > the guarantee of "referring to the same object" extends to > pointers since reference reinterpret_cast and pointer > reinterpret_cast are linked as specified in § 5.2.10/10. > Of course, in the general case there *IS* a problem when the > source lvalue is not properly aligned for the destination type > - § 5.2.10/10 doesn't explain what "referring to the same > object" is supposed to mean in this case; and § 5.2.10/7 > wouldn't apply neither. However, under the condition that > uint64_t and double have the same size, I would expect them to > have the same alignment requirement.
Alignment isn't the only potential problem. On some implementations, pointers to different types have different sizes and formats. And I believe that there is an intent that C/C++ can be implemnent on a machine with tagged addresses or actual typed data. (The Unisys Series A used something along these lines although I don't know the details. At any rate, there was only one ADD instruction, and whether it did an integer add or a floating point add depended on the address or the data.)
The standard's authors face a very difficult problem. On one hand, you do want to support reasonable expectations, but on the other, you don't want make the language impossible to implement on some architectures. I've always understood that one of the intentions for C++ is that there would be no room for a lower level language under it. And at some lower levels, such type punning is necessary: memory management software, persistency and network streaming, etc. (Not to mention implementing the C standard library -- I remember some horrible bit twiddling and type punning when I implemented modf.) On the other hand, you don't want to ban implementations on exotic architectures, and you don't want to inhibit optimization in the large majority of code which doesn't need such features. The classical solution for this sort of problem in C/C++ is undefined behavior, accompanied with more or less direct hints as to what expected behavior might be. The note at the end of §5.2.10/4 is brilliant in this respect (and IMHO really applies to reinterpret_cast in general); it says what it has to say without tying the implementors hands in any way.
> Nevertheless, I agree that the OP's code can break because of > the aliasing rules from § 3.10/15. This paragraph restricts > how a conforming program can access the stored value of an > object through an lvalue of another type - I imagine that > these restrictions exist to give the compiler better > opportunities for optimisation.
It's more fundamental than that. Suppose I use some sort of type punning such as we're discussing to put a trapping NaN into a float. The most fundamental point is that a bit pattern which is legal for one type may not be legal for another type.
I don't think that there is any question that the original posters code contains undefined behavior according to the standard. Or that there are machines on which it isn't even reasonably to expect it to work. The question is whether he can reasonably expect it to work on machines with an architecture which corresponds to what he expects.
I would say that the answer should be yes, under certain conditions. (One of those conditions would be that it might require certain compiler options to work. This is, of course a general condition for just about everything anyway, but in this case, some special options might be necessary, which aren't normally necessary for general purpose C++ code.)
> However, this paragraph doesn't restrict storing a new value > into an object through an lvalue of another object. > The most portable (for a suitable definition of "portable") > way to achieve what the OP wants seems to be > std::memcpy(&conv, &d, sizeof conv); > best accompanied with something like > BOOST_STATIC_ASSERT(sizeof conv == sizeof d);
As you say, for a suitable definition of "portable". If I understand what he is trying to do correctly, the truly portable solution involves frexp and comparisons to extract the sign, exponent and mantissa of the floating point value, and to assemble them according to what the protocol requires. This is an expensive and complicated procedure, however, and I can fully understand the desire to cut corners if all forseeable target machines acutally use the same internal format for floating point as that required by the protocol. (Having written that, I wonder if it is really that expensive and complicated. At first view, frexp() gives you all the information you need in an easy to use format, and I would expect it to be implemented using the type of type punning we're talking about, and so to be reasonably fast.)
-- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
> Niklas Matthies wrote: >> On 2005-12-05 12:29, kanze wrote: >>> Niklas Matthies wrote: : >>>> Furthermore, the result of the pointer conversion is >>>> unspecified (except that converting it back to the original >>>> type is guaranteed to yield the original pointer value) >>>> (5.2.10p7). Which means that the double * does not >>>> necessairly point at the same memory location.
>>> The standard doesn't define more, because there's not much >>> more that you can say which can be applied portably.
>> The standard could very well define that the pointer points to >> the same memory location (provided the alignment requirements >> are met as is already required).
> The standard could say that the resulting pointer points to the > same memory location, except when it doesn't. What is the > difference between this and undefined behavior?
Currently it's not undefined behavior if the alignment requirements are met, merely unspecified behavior. Instead of this unspecified behavior, the standard could define that the resulting pointer points to the same memory location, period. According to some other recent posting in this thread, the standard actually even does so.
>>> In this case, there was nothing fundamentally wrong with the >>> initial code. It makes a certain number of assumptions >>> concerning the underlying hardware, so it is definitly not >>> portable.
>> The code also makes the more important assumption that the >> compiler assumes aliasing between lvalues of unrelated types. >> In my opinion that is fundamentally wrong.
> In general I agree. In the specific case in question, however, > the aliasing was directly visible -- the compiler knew that the > programmer was playing games.
Nevertheless it's not unlikely that this has to be programmed into the compiler as an exception from the general rule, and one simply cannot assume that this has been done for all compilers.
> FWIW: historically, in K&R C (or at least Johnson's pcc), there where > three was of doing such type punning:
> 1. > union U { uint32_t raw ; float f ; } tmp ; > tmp.f = floatIn ; > rawBitsOut = tmp.raw ; : > 3. > uint32_t raw ; > *(float&)(&raw) = floatIn ; /* OR */ > raw = *(uint32_t)(&floatIn) ;
> My understanding of the situation is that the C standard also > condemns 1. Not just formally, but with the intent that it > shouldn't be expected to be supported. It's hard to be 100%, > of course, because we are talking about intent, and what one > should reasonably be able to expect from a quality > implementation, and not about strict conformance.
The correct way to do 1 is to use an array of unsigned char and size sizeof(float). One can use a macro to assemble an uint32_t from those unsigned chars (and vice versa), for example, and a good optimizer will make this as efficient as an access to an actual uint32_t.
Alternatively, the correct way to do 3 is by declaring the actual object as float and accessing it through lvalues of type unsigned char. The same comments regarding efficiency apply.
:
> So pragmatically, it would be nice to have a "portable" pragma for > this; a portable means of telling the compiler that it must take > aliasing between pointers of different types into account. I > don't quite see where such a thing fits into the standard, > however.
If anything, I'd add a new type qualifier, a sort-of opposite of C99's "restrict" qualifier.
> Falk Tannhäuser wrote: : >> Of course, in the general case there *IS* a problem when the >> source lvalue is not properly aligned for the destination type >> - § 5.2.10/10 doesn't explain what "referring to the same >> object" is supposed to mean in this case; and § 5.2.10/7 >> wouldn't apply neither. However, under the condition that >> uint64_t and double have the same size, I would expect them to >> have the same alignment requirement.
> Alignment isn't the only potential problem. On some > implementations, pointers to different types have different > sizes and formats.
I don't see how this is a problem. Size differences are subsumed by the alignment requirements, and format differences are handled by the pointer conversion operation. This is already the case for conversions from or to void *.
> And I believe that there is an intent that C/C++ can be implemnent > on a machine with tagged addresses or actual typed data.
C/C++ implementations already have to support unions, in particular pointers to union members, regardless of which union member is the "active" one at any particular point in time, so this can't be much of an actual problem.
Niklas Matthies wrote: > On 2005-12-06 10:36, kanze wrote: > > Niklas Matthies wrote: > >> On 2005-12-05 12:29, kanze wrote: > >>> Niklas Matthies wrote: > >>>> Furthermore, the result of the pointer conversion is > >>>> unspecified (except that converting it back to the > >>>> original type is guaranteed to yield the original pointer > >>>> value) (5.2.10p7). Which means that the double * does not > >>>> necessairly point at the same memory location. > >>> The standard doesn't define more, because there's not much > >>> more that you can say which can be applied portably. > >> The standard could very well define that the pointer points > >> to the same memory location (provided the alignment > >> requirements are met as is already required). > > The standard could say that the resulting pointer points to > > the same memory location, except when it doesn't. What is > > the difference between this and undefined behavior? > Currently it's not undefined behavior if the alignment > requirements are met, merely unspecified behavior. Instead of > this unspecified behavior, the standard could define that the > resulting pointer points to the same memory location, period. > According to some other recent posting in this thread, the > standard actually even does so.
The standard, for better or for worse, is trying to say that this should work when it works at the hardware level. The problem is that there is no real way to express this in standardese. Thus: vague promesses, hints of what the intent is, and formally, undefined or unspecified behavior. I think it's about the best the standard can do.
> >>> In this case, there was nothing fundamentally wrong with > >>> the initial code. It makes a certain number of > >>> assumptions concerning the underlying hardware, so it is > >>> definitly not portable. > >> The code also makes the more important assumption that the > >> compiler assumes aliasing between lvalues of unrelated > >> types. In my opinion that is fundamentally wrong. > > In general I agree. In the specific case in question, > > however, the aliasing was directly visible -- the compiler > > knew that the programmer was playing games. > Nevertheless it's not unlikely that this has to be programmed > into the compiler as an exception from the general rule, and > one simply cannot assume that this has been done for all > compilers.
agreed. We're talking about quality of implementation here.
> > FWIW: historically, in K&R C (or at least Johnson's pcc), > > there where three was of doing such type punning: > > 1. > > union U { uint32_t raw ; float f ; } tmp ; > > tmp.f = floatIn ; > > rawBitsOut = tmp.raw ; > > 3. > > uint32_t raw ; > > *(float&)(&raw) = floatIn ; /* OR */ > > raw = *(uint32_t)(&floatIn) ; > > My understanding of the situation is that the C standard > > also condemns 1. Not just formally, but with the intent > > that it shouldn't be expected to be supported. It's hard to > > be 100%, of course, because we are talking about intent, and > > what one should reasonably be able to expect from a quality > > implementation, and not about strict conformance. > The correct way to do 1 is to use an array of unsigned char > and size sizeof(float). One can use a macro to assemble an > uint32_t from those unsigned chars (and vice versa), for > example, and a good optimizer will make this as efficient as > an access to an actual uint32_t.
This is really what the discussion is all about. Looking at an object as a succession of smaller objects means taking byte order into consideration. Formally, it's not an issue, since we are doing something implementation dependant anyway. Practically, it is an issue, because in many, many cases, the only real difference on the targetted platforms is byte order; abstract the byte order, e.g. by treating a float as an unsigned long, and shifting it, and we end up with code which is "sufficiently" portable for our needs.
> Alternatively, the correct way to do 3 is by declaring the > actual object as float and accessing it through lvalues of > type unsigned char. The same comments regarding efficiency > apply.
I'm not too worried about the differences of efficiency at this level. I do want to avoid the extreme loss of efficiency which results in writing totally portable code (e.g. using frexp, etc. to extract the different fields in the float, and inserting them individually into the stream).
Although I really should measure this. I may be wrong in assuming that it is so inefficient. FWIW: the following seems to work for double:
> > So pragmatically, it would be nice to have a "portable" > > pragma for this; a portable means of telling the compiler > > that it must take aliasing between pointers of different > > types into account. I don't quite see where such a thing > > fits into the standard, however.
It probably still needs some special code to handle infinities and NaN's, and with a traits class, I think it should be possible to make it work for both double and float.
A quick test on my machine (Sun Sparc under Solaris, code compiled with g++ 3.4.3 -O3) shows that in fact the above takes from 2.5 to 3 times longer than just shifting or byte copying (and that there is no measurable difference between shifting and copying -- in every case, I used a function like the above, called with an std::back_inserter() on an std::vector< uint8_t >). But I'm not sure it matters -- even the above algorithm takes around than 1.4 microseconds per IEEE double. (And my Sparc is somewhat old, and certainly not the fastest machine around.)
On the other hand, which would you rather have to maintain: the above, something like:
uint64_t const ul = *reinterpret_cast< uint64_t const* >( &value ) ; *dest ++ = ul >> 56 ; *dest ++ = ul >> 48 ; *dest ++ = ul >> 40 ; *dest ++ = ul >> 32 ; *dest ++ = ul >> 24 ; *dest ++ = ul >> 16 ; *dest ++ = ul >> 8 ; *dest ++ = ul ; return dest ;
or code which depends somehow on the byte order?
(To be honest, the differences in both run-time and code complexity are less than I imagined before trying the experiment. And of course, if a IBM mainframe is part of your possible targets, the portable solution is the only one which will work there.)
> If anything, I'd add a new type qualifier, a sort-of opposite > of C99's "restrict" qualifier.
Sounds like a reasonable idea.
To tell the truth, I have no strong option as to whether using reinterpret_cast or using a union is best for this sort of thing; I guess the reinterpret_cast is more flexible, because it can be used on objects "in place". Currently, I think that g++ guarantees the use of a union, and requires a special option for the use of reinterpret_cast to be guaranteed. Other compilers may have different rules. All in all, I'd be surprised if a compiler didn't support some means of this; the problem is how to specify that it is wanted portably.
-- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Niklas Matthies wrote: > On 2005-12-06 11:23, kanze wrote: > > Falk Tannhäuser wrote: > >> Of course, in the general case there *IS* a problem when > >> the source lvalue is not properly aligned for the > >> destination type - § 5.2.10/10 doesn't explain what > >> "referring to the same object" is supposed to mean in this > >> case; and § 5.2.10/7 wouldn't apply neither. However, > >> under the condition that uint64_t and double have the same > >> size, I would expect them to have the same alignment > >> requirement. > > Alignment isn't the only potential problem. On some > > implementations, pointers to different types have different > > sizes and formats. > I don't see how this is a problem. Size differences are > subsumed by the alignment requirements, and format differences > are handled by the pointer conversion operation. This is > already the case for conversions from or to void *.
Different intermediate conversions can loose information, which it may not be possible to restore. Typically, this does involve alignment as well, but I wouldn't swear that this is always the case. More generally, the goal of the standard is to allow implementations on really, really wierd architectures.
> > And I believe that there is an intent that C/C++ can be > > implemnent on a machine with tagged addresses or actual > > typed data. > C/C++ implementations already have to support unions, in > particular pointers to union members, regardless of which > union member is the "active" one at any particular point in > time, so this can't be much of an actual problem.
According to the standard(s), it is undefined behavior to access any member of a union except the last assigned to. Logically, I don't see how this would be any different from the case of a reinterpret_cast, but from what I remember from discussions around the time C was being standardized, the intent was that this really should be undefined, whereas a cast should do something reasonable for the architecture in question.
In the C++ standard, there isn't the slightest hint that accessing anything but the last member assigned to in a union should work. There are vague phrases like "it is intended to be unsurprising" floating around in the description of reinterpret_cast.
Anyhow, given just how fast the truly portable solution is in this case (see my other posting), I think I'll go that way in the future.
-- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
> Niklas Matthies wrote: >> On 2005-12-06 11:23, kanze wrote: : >> > On some implementations, pointers to different types have >> > different sizes and formats.
>> I don't see how this is a problem. Size differences are >> subsumed by the alignment requirements, and format differences >> are handled by the pointer conversion operation. This is >> already the case for conversions from or to void *.
> Different intermediate conversions can loose information, which > it may not be possible to restore. Typically, this does involve > alignment as well, but I wouldn't swear that this is always the > case. More generally, the goal of the standard is to allow > implementations on really, really wierd architectures.
Since it's already guaranteed that converting the pointer back to the original type yields the original value (5.2.10p7), information loss can't be an actual issue.
>> > And I believe that there is an intent that C/C++ can be >> > implemnent on a machine with tagged addresses or actual >> > typed data.
>> C/C++ implementations already have to support unions, in >> particular pointers to union members, regardless of which >> union member is the "active" one at any particular point in >> time, so this can't be much of an actual problem.
> According to the standard(s), it is undefined behavior to access > any member of a union except the last assigned to.
I'm talking about code like:
union U { int n; double f; }; U u; u.n = 5; double * p = &u.f; // valid *p = 1.0; // also valid
Niklas Matthies wrote: > On 2005-12-06 10:36, kanze wrote: >>So pragmatically, it would be nice to have a "portable" pragma for >>this; a portable means of telling the compiler that it must take >>aliasing between pointers of different types into account. I >>don't quite see where such a thing fits into the standard, >>however.
> If anything, I'd add a new type qualifier, a sort-of opposite of C99's > "restrict" qualifier.
Shouldn't "volatile" be practically sufficient for this?
> Niklas Matthies wrote: >> On 2005-12-06 10:36, kanze wrote: >>>So pragmatically, it would be nice to have a "portable" pragma for >>>this; a portable means of telling the compiler that it must take >>>aliasing between pointers of different types into account. I >>>don't quite see where such a thing fits into the standard, >>>however.
>> If anything, I'd add a new type qualifier, a sort-of opposite of C99's >> "restrict" qualifier.
> Shouldn't "volatile" be practically sufficient for this?
Niklas Matthies <usenet-nos...@nmhq.net> wrote: > On 2005-12-08 16:19, Falk Tannh?user wrote: >> Niklas Matthies wrote: >>> If anything, I'd add a new type qualifier, a sort-of opposite of C99's >>> "restrict" qualifier.
>> Shouldn't "volatile" be practically sufficient for this?
> Good point. Yes, I believe it should.
I tried that originally on g++ 4.0.x and it didn't change the behaviour, so there are people who would not agree, unfortunately.
> Niklas Matthies <usenet-nos...@nmhq.net> wrote: >> On 2005-12-08 16:19, Falk Tannh?user wrote: >>> Niklas Matthies wrote: >>>> If anything, I'd add a new type qualifier, a sort-of opposite of C99's >>>> "restrict" qualifier.
>>> Shouldn't "volatile" be practically sufficient for this?
>> Good point. Yes, I believe it should.
> I tried that originally on g++ 4.0.x and it didn't change the behaviour, > so there are people who would not agree, unfortunately.
Niklas Matthies wrote: > On 2005-12-08 12:00, kanze wrote: > > Niklas Matthies wrote: > >> On 2005-12-06 11:23, kanze wrote: > >> > On some implementations, pointers to different types have > >> > different sizes and formats. > >> I don't see how this is a problem. Size differences are > >> subsumed by the alignment requirements, and format > >> differences are handled by the pointer conversion > >> operation. This is already the case for conversions from or > >> to void *. > > Different intermediate conversions can loose information, > > which it may not be possible to restore. Typically, this > > does involve alignment as well, but I wouldn't swear that > > this is always the case. More generally, the goal of the > > standard is to allow implementations on really, really wierd > > architectures. > Since it's already guaranteed that converting the pointer back > to the original type yields the original value (5.2.10p7), > information loss can't be an actual issue.
I'm not sure of the details, and it is possible that in all of the cases which present a problem, alignment is also an issue. What I do know is that when alignment is the only issue, as it is on a Sparc, it's fairly easy to handle, whereas in some of the other cases...
> >> > And I believe that there is an intent that C/C++ can be > >> > implemnent on a machine with tagged addresses or actual > >> > typed data. > >> C/C++ implementations already have to support unions, in > >> particular pointers to union members, regardless of which > >> union member is the "active" one at any particular point in > >> time, so this can't be much of an actual problem. > > According to the standard(s), it is undefined behavior to > > access any member of a union except the last assigned to. > I'm talking about code like: > union U { int n; double f; }; > U u; > u.n = 5; > double * p = &u.f; // valid > *p = 1.0; // also valid
But you've still only got a pointer to double. There's no issue of converting or casting a pointer here.
With regards to the more general discussion...
The more I think about it, the more I think I'm wrong. Casting the address of a double to the address of a similar sized integral type isn't a good way of streaming -- to begin with, there's no guarantee that the two use the same byte order. (I've actually worked on machines where the byte order of longs and floats was different. I'd forgotten about them here, though.) Given the discussions here, and the results of my experiments, in the future I will say that the only correct way to stream floats and doubles (regardless of the platform) is with frexp and ldexp and some bitwise manipulations. It's not that hard, and it's not that expensive in runtime, and there's no reason to try to gamble on possible platform dependant guarantees.
-- James Kanze GABI Software Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
> Niklas Matthies wrote: >> On 2005-12-08 12:00, kanze wrote: >> > Niklas Matthies wrote: >> >> On 2005-12-06 11:23, kanze wrote: : >> >> > And I believe that there is an intent that C/C++ can be >> >> > implemnent on a machine with tagged addresses or actual >> >> > typed data.
>> >> C/C++ implementations already have to support unions, in >> >> particular pointers to union members, regardless of which >> >> union member is the "active" one at any particular point in >> >> time, so this can't be much of an actual problem.
>> > According to the standard(s), it is undefined behavior to >> > access any member of a union except the last assigned to.
>> I'm talking about code like:
>> union U { int n; double f; }; >> U u; >> u.n = 5; >> double * p = &u.f; // valid >> *p = 1.0; // also valid
> But you've still only got a pointer to double. There's no issue > of converting or casting a pointer here.
You were arguing that such conversions could be problematic because of tagged addresses or typed memory. I've been trying to point out these are already problematic with what the standard currently guarantees.
Niklas Matthies <usenet-nos...@nmhq.net> wrote: > On 2005-12-09 11:08, Chris Frey wrote: >> I tried that originally on g++ 4.0.x and it didn't change the behaviour, >> so there are people who would not agree, unfortunately.
|
