Understanding Kernel-FrontEnd communication — Why

Question

EDIT: Just a confirmation whether you can reproduce this or not would be useful. Only a single computer is needed to try this (no remote connection necessary).

Update It seems other can't reproduce this on Mac or Win7, so it's either WinXP-specific or specific to my machine. At this point I'm giving up.

---

It would be good to have a tutorial on how the Front End and the Kernel communicate, so we can debug remote kernel issues. Any such general answers (or links to tutorials elsewhere) are most welcome. I am already aware of Remote Kernel Strategies by Sascha Kratky.

Now the practical question: When I connect to a remote machine in any way (e.g. using remote kernel strategies), the front end will sooner or later freeze up. How can I prevent this?

Here's how to reproduce the problem on a single machine (no remote connection needed):

1. Create a new kernel configuration (Evaluation menu -> Kernel Configuration Options...)

   Select Advanced Options.

   Arguments to MLOpen:

   -LinkName "8000@YOUR-IP-HERE,8001@YOUR-IP-HERE" -LinkMode Listen 
   -LinkProtocol TCPIP -LinkHost "YOUR-IP-HERE" -LinkOptions MLDontInteract
   

   (Replace YOUR-IP-HERE with your computer's IP address. On Windows you can get this by typing ipconfig in a command window.)

   Shell command to launch kernel: leave empty (we will do it manually)

2. Open a new notebook, set the notebook's kernel to the one you just set up, and evaluate something (1+1).

3. Now we need to launch the kernel manually, before the connection times out. So start a new kernel (math) in command line mode, and evaluate the following:

   link=LinkConnect["8000,8001",LinkProtocol->"TCPIP"] (* connect to front end link *)
   
   $ParentLink = link; (* set the front end link as parent link *)
   

   Now the connection is established, and everything appears to work correctly (save for the one message in the front end's pink message window)

4. Evaluate Manipulate[n, {n, 0, 100, 1}]. First, this appears to work too. Now play with the slider. Sooner or later the front end will freeze and needs to be killed. For me, dragging the slider up and down for ~10 seconds without releasing it always suffices.

Why does the Front End freeze? How can I create a TCPIP-mode connection between the front end and the kernel in a way that everything will be working correctly?

---

Some notes:

• I'm on Windows XP.
• The problem never appears if I use LinkProtocol -> "SharedMemory".
• Aborting calculations using Evaluation -> Abort Evaluation does work correctly.
• I have verified using Links[] that MathLink`$PreemptiveLink and MathLink`$ServiceLink are created and LinkConnectedQ[] returns True on them.

Again, note that the Front End will freeze only after dragging the Manipualate slider continuously for a few seconds without releasing it.

Link to same question on MathGroup.

_{Related: Firewall settings for remote kernel to work (Mathematica)}

Answer 1

(not an answer, but hard to write this as comment)

fyi, there is a little bit on information about how the frontend talks to the kernel here http://reference.wolfram.com/mathematica/tutorial/AdvancedDynamicFunctionality.html

about half way down, it says

The front end and kernel communicate with each other through several 
MathLink connections, known as the main link, the preemptive link, 
and the service link. The main and preemptive links are pathways by 
which the front end can send evaluation requests to the kernel, and 
the kernel can respond with results. The service link works in reverse, 
with the kernel sending requests to the front end.

and more there. May be that will help. Also, same information is in the PDF file on page 31:

http://www.wolfram.com/learningcenter/tutorialcollection/DynamicInteractivity/

The actual messaging protocol between the kernel and front end must be very advanced.