Purpose
I am writing a network program in C (specifically gnu89) and I would like to simplify things by reinterpreting a certain struct X as big array of bytes (a.k.a. char), sending the bytes over the network, and reinterpreting them as struct X on the other side. To this end I have decided to use gcc's __attribute__((__packed__ )). I have done my best to ensure that this is done correctly (i.e. I've accounted for endianness and other related issues).
Question
Other than guaranteeing that struct X is as small as possible, does gcc guarantee that a struct defined with __attribute__((__packed__ )) retains the original ordering? I've done a fair amount of searching and I have yet to find any documentation on whether or not this guarantee exists.
Notes
It is safe to assume that both the sender and receiver will encounter no portability issues (e.g. sizeof(int) on the server is equal to sizeof(int) on the client).
Assuming that you are asking whether the struct members will retain the order specified in their definition, the answer is yes. The Standard requires that successive members have increasing addresses:
Section §6.7.2.1p13:
Within a
structure object, the non-bit-field
members and the units in which
bit-fields reside have addresses that
increase in the order in which they
are declared.
and the documentation for the packed attribute clearly states that only padding/alignment is affected:
The packed attribute specifies that a
variable or structure field should
have the smallest possible
alignment—one byte for a variable, and
one bit for a field, unless you
specify a larger value with the
aligned attribute.
Yes, __attribute__((packed)) (no need for second set of underscores) is a correct way to implement binary (i.e. non-text) network protocols. There will be no gaps between the elements.
However you should understand that packed not only packs the structure, but also:
- makes its required alignment one byte, and
- makes sure that its members, which may get misaligned by the packing and by lack of alignment requirement of the struct itself, are read and written correctly, i.e. misalignment of its fields is dealt with in software by the compiler.
However, the compiler will only deal with misalignment if you access the struct members directly. You should never make a pointer to a member of a packed struct (except when you know the member's required alignment is 1, like char or another packed struct). The following C code demonstrates the issue:
#include <stdio.h>
#include <inttypes.h>
#include <arpa/inet.h>
struct packet {
uint8_t x;
uint32_t y;
} __attribute__((packed));
int main ()
{
uint8_t bytes[5] = {1, 0, 0, 0, 2};
struct packet *p = (struct packet *)bytes;
// compiler handles misalignment because it knows that
// "struct packet" is packed
printf("y=%"PRIX32", ", ntohl(p->y));
// compiler does not handle misalignment - py does not inherit
// the packed attribute
uint32_t *py = &p->y;
printf("*py=%"PRIX32"\n", ntohl(*py));
return 0;
}
On an x86 system (which does not enforce memory access alignment), this will produce
y=2, *py=2
as expected. On the other hand on my ARM Linux board, for example, it produced the seemingly wrong result
y=2, *py=1
Yes.
However, using __attribute__((__packed__)) is not a good way to do what you are doing.
- It doesn't resolve byte order issues
- Access to the structure will be slow
- Although the popularity of gcc has led to a situation where other compilers often implement gcc extensions, using this compiler-specific extension means that you do not have a conforming C program. This means that if another compiler or even a future gcc changes packed or doesn't implement it at all, you are out of luck and can't even complain to anyone. Gcc could drop it tomorrow and still be a C99 compiler. (Ok, that exact thing is unlikely.) Most of us try to write conforming programs not because we have some abstract hostility to using vendor software or desire some academic standard of code purity, but rather because we know that only conforming language features have a precise and common specification, so it is far far easier to depend on our code doing the right thing from day to day and system to system if we do it that way.
- You are reinventing the wheel; this problem has already been solved in a standard-conforming way: see YAML, XML, and JSON.
- If it's such a low-level protocol that YAML, XML, and JSON are not available, you really should take individual fundamental types, apply your host's version of hton?() and ntoh?(), and memcpy() them to an output buffer. I realize that there is a long tradition of reading and writing straight from structures, but I've also spent a long time fixing that code later when it was moved from 32-bit to 64-bit environments...
Based on what you're trying to do, I'd highly encourage you to also used fixed-size data types (ie. uint32_t, int16_t, etc) that are found in stdint.h. Using fixed-size data types will prevent you from having to do things like the following:
struct name
{
short field : 8;
};
We use this technique frequently to convert messages between a byte array and a structure, and have never encountered problems with it. You may have to perform endianness conversion yourself, but field order isn't a problem. If you have any concerns about data type sizes, you can always specify field size like so:
struct foo
{
short someField : 16 __attribute__ ((packed));
};
This guarantees that someField will be stored as 16 bits and will not be rearranged or altered to fit byte boundaries.
Yes, C has a guarantee that struct elements won't be reordered. (There may be extensions or fancy optimization systems that might change this, but not by default in gcc.)
