Calculating CPU frequency in C with RDTSC always r

Question

The following piece of code was given to us from our instructor so we could measure some algorithms performance:

#include <stdio.h>
#include <unistd.h>

static unsigned cyc_hi = 0, cyc_lo = 0;

static void access_counter(unsigned *hi, unsigned *lo) {
    asm("rdtsc; movl %%edx,%0; movl %%eax,%1"
    : "=r" (*hi), "=r" (*lo)
    : /* No input */
    : "%edx", "%eax");
}

void start_counter() {
    access_counter(&cyc_hi, &cyc_lo);
}

double get_counter() {
    unsigned ncyc_hi, ncyc_lo, hi, lo, borrow;
    double result;

    access_counter(&ncyc_hi, &ncyc_lo);

    lo = ncyc_lo - cyc_lo;
    borrow = lo > ncyc_lo;
    hi = ncyc_hi - cyc_hi - borrow;

    result = (double) hi * (1 << 30) * 4 + lo;

    return result;
}

However, I need this code to be portable to machines with different CPU frequencies. For that, I'm trying to calculate the CPU frequency of the machine where the code is being run like this:

int main(void)
{
    double c1, c2;

    start_counter();

    c1 = get_counter();
    sleep(1);
    c2 = get_counter();

    printf("CPU Frequency: %.1f MHz\n", (c2-c1)/1E6);
    printf("CPU Frequency: %.1f GHz\n", (c2-c1)/1E9);

    return 0;
}

The problem is that the result is always 0 and I can't understand why. I'm running Linux (Arch) as guest on VMware.

On a friend's machine (MacBook) it is working to some extent; I mean, the result is bigger than 0 but it's variable because the CPU frequency is not fixed (we tried to fix it but for some reason we are not able to do it). He has a different machine which is running Linux (Ubuntu) as host and it also reports 0. This rules out the problem being on the virtual machine, which I thought it was the issue at first.

Any ideas why this is happening and how can I fix it?

Answer 1

As for VMWare, take a look at the time keeping spec (PDF Link), as well as this thread. TSC instructions are (depending on the guest OS):

• Passed directly to the real hardware (PV guest)
• Count cycles while the VM is executing on the host processor (Windows / etc)

Note, in #2 the while the VM is executing on the host processor. The same phenomenon would go for Xen, as well, if I recall correctly. In essence, you can expect that the code should work as expected on a paravirtualized guest. If emulated, its entirely unreasonable to expect hardware like consistency.

Answer 2

I can't say for certain what exactly is wrong with your code, but you're doing quite a bit of unnecessary work for such a simple instruction. I recommend you simplify your rdtsc code substantially. You don't need to do 64-bit math carries your self, and you don't need to store the result of that operation as a double. You don't need to use separate outputs in your inline asm, you can tell GCC to use eax and edx.

Here is a greatly simplified version of this code:

#include <stdint.h>

uint64_t rdtsc() {
    uint64_t ret;

# if __WORDSIZE == 64
    asm ("rdtsc; shl $32, %%rdx; or %%rdx, %%rax;"
        : "=A"(ret)
        : /* no input */
        : "%edx"
    );
#else
    asm ("rdtsc" 
        : "=A"(ret)
    );
#endif
    return ret;
}

Also you should consider printing out the values you're getting out of this so you can see if you're getting out 0s, or something else.

Answer 3

You forgot to use volatile in your asm statement, so you're telling the compiler that the asm statement produces the same output every time, like a pure function. (volatile is only implicit for asm statements with no outputs.)

This explains why you're getting exactly zero: the compiler optimized end-start to 0 at compile time, through CSE (common-subexpression elimination).

See my answer on Get CPU cycle count? for the __rdtsc() intrinsic, and @Mysticial's answer there has working GNU C inline asm, which I'll quote here:

// prefer using the __rdtsc() intrinsic instead of inline asm at all.
uint64_t rdtsc(){
    unsigned int lo,hi;
    __asm__ __volatile__ ("rdtsc" : "=a" (lo), "=d" (hi));
    return ((uint64_t)hi << 32) | lo;
}

This works correctly and efficiently for 32 and 64-bit code.

Answer 4

Okay, since the other answer wasn't helpful, I'll try to explain on more detail. The problem is that a modern CPU can execute instructions out of order. Your code starts out as something like:

rdtsc
push 1
call sleep
rdtsc

Modern CPUs do not necessarily execute instructions in their original order though. Despite your original order, the CPU is (mostly) free to execute that just like:

rdtsc
rdtsc
push 1
call sleep

In this case, it's clear why the difference between the two rdtscs would be (at least very close to) 0. To prevent that, you need to execute an instruction that the CPU will never rearrange to execute out of order. The most common instruction to use for that is CPUID. The other answer I linked should (if memory serves) start roughly from there, about the steps necessary to use CPUID correctly/effectively for this task.

Of course, it's possible that Tim Post was right, and you're also seeing problems because of a virtual machine. Nonetheless, as it stands right now, there's no guarantee that your code will work correctly even on real hardware.

Edit: as to why the code would work: well, first of all, the fact that instructions can be executed out of order doesn't guarantee that they will be. Second, it's possible that (at least some implementations of) sleep contain serializing instructions that prevent rdtsc from being rearranged around it, while others don't (or may contain them, but only execute them under specific (but unspecified) circumstances).

What you're left with is behavior that could change with almost any re-compilation, or even just between one run and the next. It could produce extremely accurate results dozens of times in a row, then fail for some (almost) completely unexplainable reason (e.g., something that happened in some other process entirely).

Answer 5

hmmm I'm not positive but I suspect the problem may be inside this line:

result = (double) hi * (1 << 30) * 4 + lo;

I'm suspicious if you can safely carry out such huge multiplications in an "unsigned"... isn't that often a 32-bit number? ...just the fact that you couldn't safely multiply by 2^32 and had to append it as an extra "* 4" added to the 2^30 at the end already hints at this possibility... you might need to convert each sub-component hi and lo to a double (instead of a single one at the very end) and do the multiplication using the two doubles