Removing invalid characters from XML before serial

Question

I'm trying to store user-input in an XML document on the client-side (javascript), and transmit that to the server for persistence.

One user, for example, pasted in text that included an STX character (0x2). The XMLSerializer did not escape the STX character, and therefore, did not serialize to well-formed XML. Or perhaps the .attr() call should have escaped the STX character, but in either case, invalid XML was produced.

I'm finding the output of in-browser XMLSerializer() isn't always well-formed, (and doesn't even satisfy the browser's own DOMParser()

This example shows that the STX character is not properly encoded by XMLSerializer():

> doc = $.parseXML('<?xml version="1.0" encoding="utf-8" ?>\n<elem></elem>');
    #document
> $(doc).find("elem").attr("someattr", String.fromCharCode(0x2));
    [ <elem someattr=​"">​</elem>​ ]
> serializedDoc = new XMLSerializer().serializeToString(doc);
    "<?xml version="1.0" encoding="utf-8"?><elem someattr=""/></elem>"
> $.parseXML(serializedDoc);
    Error: Invalid XML: <?xml version="1.0" encoding="utf-8"?><elem someattr=""/></elem>

How should I construct an XML document in-browser (with params determined by arbitrary user-input) such that it will always be well-formed (everything properly escaped)? I don't need to support IE8 or IE7.

(And yes, I do validate the XML on the server side, but if the browser hands the server a document that is not well-formed, the best the server can do is reject it, which isn't that helpful to the poor user)

Answer 1

Here's a function sanitizeStringForXML() which can either be used to cleanse strings before assignment, or a derivative function removeInvalidCharacters(xmlNode) which can be passed a DOM tree and will automatically sanitize attributes and textNodes so they are safe to store.

var stringWithSTX = "Bad" + String.fromCharCode(2) + "News";
var xmlNode = $("<myelem/>").attr("badattr", stringWithSTX);

var serializer = new XMLSerializer();
var invalidXML = serializer.serializeToString(xmlNode);

// Now cleanse it:
removeInvalidCharacters(xmlNode);
var validXML = serializer.serializeToString(xmlNode);

I based this on a list of characters from the non-restricted characters section of this wikipedia article, but the supplementary planes require 5-hex-digit unicode characters, and the Javascript regex does not include a syntax for this, so for now, I'm just stripping them out (you aren't missing too much...):

// WARNING: too painful to include supplementary planes, these characters (0x10000 and higher) 
// will be stripped by this function. See what you are missing (heiroglyphics, emoji, etc) at:
// http://en.wikipedia.org/wiki/Plane_(Unicode)#Supplementary_Multilingual_Plane
var NOT_SAFE_IN_XML_1_0 = /[^\x09\x0A\x0D\x20-\xFF\x85\xA0-\uD7FF\uE000-\uFDCF\uFDE0-\uFFFD]/gm;
function sanitizeStringForXML(theString) {
    "use strict";
    return theString.replace(NOT_SAFE_IN_XML_1_0, '');
}

function removeInvalidCharacters(node) {
    "use strict";

    if (node.attributes) {
        for (var i = 0; i < node.attributes.length; i++) {
            var attribute = node.attributes[i];
            if (attribute.nodeValue) {
                attribute.nodeValue = sanitizeStringForXML(attribute.nodeValue);
            }
        }
    }
    if (node.childNodes) {
        for (var i = 0; i < node.childNodes.length; i++) {
            var childNode = node.childNodes[i];
            if (childNode.nodeType == 1 /* ELEMENT_NODE */) {
                removeInvalidCharacters(childNode);
            } else if (childNode.nodeType == 3 /* TEXT_NODE */) {
                if (childNode.nodeValue) {
                    childNode.nodeValue = sanitizeStringForXML(childNode.nodeValue);
                }
            }
        }
    }
}

Note that this only removes invalid characters from nodeValues of attributes and textNodes. It does not check tag names or attribute names, comments, etc etc.