What exactly is the halting problem?

Question

Whenever people ask about the halting problem as it pertains to programming, people respond with "If you just add one loop, you've got the halting program and therefore you can't automate task"

Makes sense. If your program has an infinite loop, then when your program is running, you have no way of knowing whether the program is still crunching input, or if it is just looping infinitely.

But some of this seems counter intuitive. What if I was writing a halting problem solver, which takes source code as its input. rascher@localhost$ ./haltingSolver source.c

If my code (source.c) looks like this:

for (;;) {  /* infinite loop */  }

It seems like it'd be pretty easy for my program to see this. "Look the loop, and look at the condition. If the condition is just based on literals, and no variables, then you always know the outcome of the loop. If there are variables (eg while (x < 10)), see if those variables are ever modified. If not, then you always know the outcome of the loop."

Granted, these checks would not be trivial (calculating pointer arithmetics, etc) but it does not seem impossible. eg:

int x = 0
while (x < 10) {}

could be detected. along with - albeit not trivially:

int x = 0
while (x < 10)
{
   x++;
   if (x == 10)
   {
      x = 0
   }
}

Now what about user input? That is the kicker, that is what makes a program unpredictable.

int x = 0;
while (x < 10) 
{
   scanf("%d", &x); /* ignoring infinite scanf loop oddities */
}

Now my program can say: "If the user enters a 10 or greater, the program will halt. On all other input, it will loop again."

Which means that, even with hundreds of inputs, one ought to be able to list the conditions on which the program will stop. Indeed, when I write a program, I always make sure someone has the ability to terminate it! I am not saying that the resulting list of conditions is trivial to create, but it doesn't seem impossible to me. You could take input from the user, use them to calculate pointer indexes, etc - but that just adds to the number of conditions to ensure the program will terminate, doesn't make it impossible to enumerate them.

So what exactly is the halting problem? What am I not understanding about the idea that we cannot write a problem to detect infinite loops? Or, why are "loops" such an oft-cited example?

UPDATE

So, let me change the question a little bit: what is the halting problem as it applies to computers? And then I will respond to some of the comments:

Many people have said that the program must be able to deal with "any arbitrary input." But in computers, there isn't ever any arbitrary input. If I only input a single byte of data, than I only have 2^8 possible inputs. So, as an example:

int c = getchar()

switch (c) {
   case 'q':
      /* quit the program */
}

All of the sudden, I have just accounted for all of the possibilities. If c has the bit pattern 0x71, it does one thing. For all other patterns, it does something else. Even a program that accepts arbitrary string input is never really "arbitrary", since resources are finite, which means that while the theory of "arbitrary" applies... it isn't exactly one-to-one with the practice.

The other example people cited is this:

while (n != 1)
    if (n & 1 == 1) 
        n = 3 * n + 1;
    else 
        n /= 2;

If n is a 32-bit integer... then I can visually tell you whether or not this will halt.

I guess this edit isn't asking anything, but the most convincing example I've seen is this one:

Assume that you have your magical program/method to determine that a program halts.

public bool DeterminesHalt(string filename, string[] args){
    //runs whatever program you tell it do, passing any args
    //returns true if the program halts, false if it doesn't
}

Now lets say we write a small piece of code such as...

public static void Main(string[] args){
    string filename = Console.ReadLine(); //read in file to run from user
    if(DeterminesHalt(filename, args))
        for(;;);
    else
        return;
}

So for this example, we can write a program to do the exact opposite of our magical halting method does. If we somehow determine that a given program will halt, we just hop into an infinite loop; otherwise if we determine that the program is in an infinite loop, we end the program.

Then again, if you intentionally write a program which contains an infinite loop... "solving the halting problem" is kind of moot, isn't it?

Answer 1

To solve the halting problem, you'd have to develop an algorithm that could determine whether any arbitrary program halts for any arbitrary input, not just the relatively simple cases in your examples.

Answer 2

A lot of interesting specific examples/analogies so far. If you want to read deeper into the background, there's a good book on Turing's original paper, The Annotated Turing, by Charles Petzold.

In a related, sideways-sorta, vein, there's a really neat essay up on the web, Who Can Name the Bigger Number? which brushes on Turing machines and Ackermann functions.

Answer 3

"If you just add one loop, you've got the halting program and therefore you can't automate task"

Sounds like someone over generalizing the application of the halting problem. There are plenty of particular loops that you can prove terminate. There exists research that can perform termination checking for wide classes of programs. For instance in Coq you are limited to programs that you can prove terminate. Microsoft has a research project called Terminator that uses various approximations to prove that programs will terminate.

But, remember, the halting problem isn't just about toy examples. Neither of those solves the general 'halting problem', because they don't work for every program.

The problem is that the halting problem says that there exist programs that you have no way to know if they will terminate without running them, which means that you may never get done deciding if they halt.

An example of a program that may or may not halt (in Haskell):

collatz 1 = ()
collatz !n | odd n     = collatz (3 * n + 1)
           | otherwise = collatz (n `div` 2)

or in something more accessible:

while (n != 1)
    if (n & 1 == 1) 
        n = 3 * n + 1;
    else 
        n /= 2;

Given every integer >= 1, will this program halt? Well, it has worked so far, but there is no theorem that says it will halt for every integer. We have a conjecture due to Lothar Collatz that dates back to 1937 that it holds, but no proof.

Answer 4

The significance of the halting problem does not lie in the importance of the problem itself; on the contrary, automated testing would be of little practical use in software engineering (proving that a program halts does not prove that it is correct, and in any case the hypothetical algorithm only provides proof for a given finite input, whereas a real-life software developer would be more interested in a test for all possible finite inputs).

Rather, the halting problem was one of the first to be proven undecidable, meaning that no algorithm exists which works in the general case. In other words, Turing proved more than 70 years ago that there are some problems which computers cannot solve -- not just because the right algorithm has not yet been found, but because such an algorithm cannot logically exist.