How do i accept MathML?

Question

i discovered today that Windows 7 comes with a very impressive MathPanel utility, for performing handwriting recognition of equations:

Which is fine. (Here i've entered the formula for the part of the sRGB color space gamma conversion)

But now i don't seem to be able to do anything with it.

There is an Insert button. i would assume that clicking Insert would insert it into the application that is active behind it (much like the On-Screen Keyboard works):

Except i assume it would operate as a Paste operation.

i can find no information in the help on what is required by an application to make it work. There is no mention of any special API some software must support.

Nor can i find any information on MSDN about what special API is required to accept the insertion of an equation.

What API, registration, callback, listener, message, COM Object do i have to implement so that i will receive MathPanel input?

The only reason i mention MathML is because an answer on SuperUser mentioned MathML:

Theoretically, any app that supports MathML (Mathematical Markup Language) can be used with the Windows 7 Math Input Panel. The Math Input Panel only works with programs that support MathML. Here are a few such apps: StarOffice, OpenOffice, Opera and Maple.

Well how do i make my program support MathML?

As far as i know MathML is a markup language; not a Windows API. It would be like saying, "How do i make my program support HTML?" Html is text, and you can paste it anywhere.

MathPad refuses to paste unless i "support" MathML?

---

Update

Inspecting the IDataObject on the clipboard after clicking Insert, i see two formats available (neither of which are text, which explains why i do not get any markup):

Format 1:

     CLIPFORMAT cfFormat: "MathML Presentation" (49839)
PDVTargetDevice ptd:      0x00000000
          DWORD dwAspect: DVASPECT_CONTENT
          DWORD lindex:   -1
          DWORD tymed:    1  (TYMED_HGLOBAL)

Format 2:

     CLIPFORMAT cfFormat:"MathML" (49838)
PDVTargetDevice ptd:      0x00000000
          DWORD dwAspect: DVASPECT_CONTENT
          DWORD lindex:   -1
          DWORD tymed:    1  (TYMED_HGLOBAL)

So at least now i have some clipboard formats:

• "MathML Presentation"
• "MathML"

i still cannot find anything on MSDN about either clipboard format.

Answer 1

Spying on messages sent to my window, looks like the Math Input Panel application sends a Ctrl+V:

• WM_KEYDOWN (0x11) VK_CONTROL
• WM_KEYDOWN (0x56) V key
• WM_CHAR (0x16)
• WM_KEYUP (0x11) VK_CONTROL
• WM_KEYUP (0x56) V key

So you need to recognize that someone's trying to hit Ctrl+V. Then you must extract the contents.

First register the three clipboard formats:

Handle CF_MathML_Presentation = RegisterClipboardFormat("MathML Presentation");
Handle CF_MathML_Content = RegisterClipboardFormat("MathML Content");
Handle CF_MathML = RegisterClipboardFormat("MathML");

Note: Appendix B of the W3C's Mathematical Markup Language (MathML) Version 3.0 documents the Windows clipboard format names to be registered:

• Generic MathML Windows Clipboard Name: MathML
• Presentation MathML Windows Clipboard Name: MathML Presentation
• Content MathML Windows Clipboard Name: MathML Content

Then get a handle on the IDataObject in the clipboard:

IDataObject dataObject;
OleGetClipboard(dataObject);

Then enumerate all the formats, looking for the one you like:

IEnumFORMATETC enum;
dataObject.EnumFormatEtc(DATADIR_GET, out enum);

String mathXml = "";

foreach (FormatEtc format in enum)
{
    if (format.cfFormat = CF_MathML_Presentation) ||
       (format.cfFormat = CF_MathML_Content) ||
       (format.cfFormat = CF_MathML)
    {
        //We know how to handle these formats:
        STGMEDIUM medium;
        dataObject.GetData(format.cfFormat, out medium);

        mathXml = GetStringFromStorageMedium(medium); //handles all the nasty HGlobal/IStream/IStorage nonsense
    }
}

ShowMessage(mathXml); //tada!

---

Microsoft also allows you to program the Math Input COM object:

//Create the COM object
IMathInputControl mathInputControl = CreateComObject(CLSID_MathInputControl);
mathInputControl.Show();

You can then create an object that receives the notification events:

class MathEvents : _IMathInputControlEvents
{
    public HRESULT Insert(String mathXml)
    {
       //Notifies the event handler when the Insert button is clicked.
       MessageBox.Show(mathXml);
       return S_OK;
    }

    public HRESULT Clear()
    {
       //Notifies the event handler when the Clear button is clicked.      
       return S_OK;
    }

    public HRESULT Close()
    {
       //Notifies the event handler when the Close button is clicked.
       return S_OK;
    }

    public HRESULT PaintHRESULT Paint(LONG_PTR hdc, LONG Left, LONG Top, LONG Right, LONG Bottom, LONG Element, LONG State)
    {
       //Notifies the event handler when the buttons and background of the control require painting.
       return S_OK;           
    }

The missing ingredient is how to give mathInputControl a reference to our callback object.

That's super-secret complicated COM code, involving ConnectionPointContainer, andAdvise`, which cannot be done from C#.

But you don't need to, you can just use Ctrl+V.

Answer 2

I think it is officially called the "Math Input Panel" (MIP). The MathType product supports it and provides a menu item to run it. As other replies here mention, clicking MIP's Insert button sends a Ctrl-V to the window underneath it. If that window supports that keyboard shortcut and handles MathML, then it will work.

While the MathML Clipboard Format is the recommended way to handle MathML input on a paste, if you are implementing MathML support in your app you should also accept MathML text offered as CF_UNICODETEXT. Some apps support copying MathML to the clipboard but do not seem to know about the MathML Clipboard Format. Of course, your paste code will have to sniff the text to identify MathML as opposed to regular non-MathML text. You should also consider accepting a drag-and-drop as well as a paste.

The Math Input Control is a related but slightly different configuration of MIP. If I remember correctly, it lacks the history and some other features of the MIP. We started working with it for MathType and quickly realized that there's no advantage to it. You should just ignore it and just support paste and drag-and-drop of MathML. Add a Math Input Panel menu item if it makes sense for your app.

Answer 3

The clipboard format "MathML Presentation" actually contains text; I've tried this before using the Windows API GetClipboardData().

Moreover, if you copy a MathML fragment to the clipboard as plain text (e.g., CF_TEXT) and then paste it to a Word document, you will get plain text also, i.e., Word will NOT interpret it as Presentation MathML.

For Word to do so, you have to copy it as both CF_TEXT and as "MathML Presentation". To obtain the ID of the latter, try registering "MathML Presentation" as a clipboard format as suggested by Ian Boyd. Windows will return its clipboard format ID; use this ID with SetClipboardData().