The Java Docs for Thread.stop() make it sound like the world will end if you ever call Thread.stop().

Deprecated. This method is inherently unsafe. Stopping a thread with Thread.stop causes it to unlock all of the monitors that it has locked (as a natural consequence of the unchecked ThreadDeath exception propagating up the stack). If any of the objects previously protected by these monitors were in an inconsistent state, the damaged objects become visible to other threads, potentially resulting in arbitrary behavior. Many uses of stop should be replaced by code that simply modifies some variable to indicate that the target thread should stop running. The target thread should check this variable regularly, and return from its run method in an orderly fashion if the variable indicates that it is to stop running. If the target thread waits for long periods (on a condition variable, for example), the interrupt method should be used to interrupt the wait. For more information, see Why are Thread.stop, Thread.suspend and Thread.resume Deprecated?.

If, for example, I'm running a method in a 3rd party, closed source library and I need to guarantee it ends in 5 minutes or less, is there any other way? What are the real-world consequences of using Thread.stop() and how can I identify if they happen?

Never!

Under no circumstances should you use Thread.stop().

If you think you need to call Thread.stop() don't! It will be wrong.

Thread.stop() should not be called under any circumstances.

Please do not call Thread.stop() - even if you are being threatened with death.

I donate this answer to community wiki please - others - join me in admonishing those heathens for the crime of calling Thread.stop() by editing this answer and adding your prayer that no mortal should call Thread.stop().

If you really want a process to be killed after 5 minutes - use a Process that can be destroyed. This will protect the main JVM from any potential corruption that might occur if a thread is terminated using Thread.stop(). However, this may leave files in a corrupted state, if the child Process was in the middle of I/O. One way to defend against that situation is to make the Process the client of a database that can roll back in-progress transactions.

In general, terminating something uncooperatively and safely is very, very difficult.