Where to find the complete definition of off_t typ

Question

I am sending file from client to server using TCP. To mark the end of the file I like to send file size before the actual data. So I use stat system call to find the size of the file. This is of type off_t. I like to know how many bytes it occupies so that I can read it properly on the server side. It is defined in the <sys/types.h>. But I do not understand the definition. It just defines __off_t or _off64_t to be off_t. Where to look for __off_t? Also is it convention that __ is prefixed for most of the things in header files and scares me when I read header files to understand better. How to read a header file better?

#ifndef __off_t_defined
# ifndef __USE_FILE_OFFSET64
typedef __off_t off_t;
# else
typedef __off64_t off_t;
# endif
# define __off_t_defined
#endif 

Answer 1

If you are having trouble tracing the definitions, you can use the preprocessed output of the compiler which will tell you all you need to know. E.g.

$ cat test.c
#include <stdio.h>
$ cc -E test.c | grep off_t
typedef long int __off_t;
typedef __off64_t __loff_t;
  __off_t __pos;
  __off_t _old_offset;
typedef __off_t off_t;
extern int fseeko (FILE *__stream, __off_t __off, int __whence);
extern __off_t ftello (FILE *__stream) ;

If you look at the complete output you can even see the exact header file location and line number where it was defined:

# 132 "/usr/include/bits/types.h" 2 3 4


typedef unsigned long int __dev_t;
typedef unsigned int __uid_t;
typedef unsigned int __gid_t;
typedef unsigned long int __ino_t;
typedef unsigned long int __ino64_t;
typedef unsigned int __mode_t;
typedef unsigned long int __nlink_t;
typedef long int __off_t;
typedef long int __off64_t;

...

# 91 "/usr/include/stdio.h" 3 4
typedef __off_t off_t;

Answer 2

Since this answer still gets voted up, I want to point out that you should almost never need to look in the header files. If you want to write reliable code, you're much better served by looking in the standard. A better question than "how is off_t defined on my machine" is "how is off_t defined by the standard?". Following the standard means that your code will work today and tomorrow, on any machine.

In this case, off_t isn't defined by the C standard. It's part of the POSIX standard, which you can browse here.

Unfortunately, off_t isn't very rigorously defined. All I could find to define it is on the page on sys/types.h:

blkcnt_t and off_t shall be signed integer types.

This means that you can't be sure how big it is. If you're using GNU C, you can use the instructions in the answer below to ensure that it's 64 bits. Or better, you can convert to a standards defined size before putting it on the wire. This is how projects like Google's Protocol Buffers work (although that is a C++ project).

---

So, I think "where do I find the definition in my header files" isn't the best question. But, for completeness here's the answer:

You'll find the definition in bits/types.h (as a comment says at the top, never directly include this file), but it's obscured a bit in a bunch of macros. An alternative to trying to unravel them is to look at the preprocessor output:

#include <stdio.h>
#include <sys/types.h>

int main(void) {
  off_t blah;

  return 0;
}

And then:

$ gcc -E sizes.c  | grep __off_t
typedef long int __off_t;
....

However, if you want to know the size of something, you can always use the sizeof() operator.

Edit: Just saw the part of your question about the __. This answer has a good discussion. The key point is that names starting with __ are reserved for the implementation (so you shouldn't start your own definitions with __).

Answer 3

As the "GNU C Library Reference Manual" says

off_t
    This is a signed integer type used to represent file sizes. 
    In the GNU C Library, this type is no narrower than int.
    If the source is compiled with _FILE_OFFSET_BITS == 64 this 
    type is transparently replaced by off64_t.

and

off64_t
    This type is used similar to off_t. The difference is that 
    even on 32 bit machines, where the off_t type would have 32 bits,
    off64_t has 64 bits and so is able to address files up to 2^63 bytes
    in length. When compiling with _FILE_OFFSET_BITS == 64 this type 
    is available under the name off_t.

Thus if you want reliable way of representing file size between client and server, you can:

1. Use off64_t type and stat64() function accordingly (as it fills structure stat64, which contains off64_t type itself). Type off64_t guaranties the same size on 32 and 64 bit machines.
2. As was mentioned before compile your code with -D_FILE_OFFSET_BITS == 64 and use usual off_t and stat().

3. Convert off_t to type int64_t with fixed size (C99 standard). Note: (my book 'C in a Nutshell' says that it is C99 standard, but optional in implementation). The newest C11 standard says:

   7.20.1.1 Exact-width integer types
   1 The typedef name intN_t designates a signed integer type with width N ,
   no padding bits, and a two’s complement representation. Thus, int8_t 
   denotes such a signed integer type with a width of exactly 8 bits.
   without mentioning.
   

   And about implementation:

   7.20 Integer types <stdint.h>
   ... An implementation shall provide those types described as ‘‘required’’,
   but need not provide any of the others (described as ‘‘optional’’).
   ...
   The following types are required:
   int_least8_t  uint_least8_t
   int_least16_t uint_least16_t
   int_least32_t uint_least32_t
   int_least64_t uint_least64_t
   All other types of this form are optional.
   

Thus, in general, C standard can't guarantee types with fixed sizes. But most compilers (including gcc) support this feature.

Answer 4

If you are writing portable code, the answer is "you can't tell", the good news is that you don't need to. Your protocol should involve writing the size as (eg) "8 octets, big-endian format" (Ideally with a check that the actual size fits in 8 octets.)