XML - Referencing Other XML Files

Question

I'm new to XML, so this may be a fairly easy question to answer. I was wondering if there is a standard way of referencing external XML files from within other XML files. Let me give an example. Say you have a file which defines a single object that holds a large amount of data:

<person>
    <name>John</name>
    <age>18</age>
    <hair>Brown</hair>
    <eyes>Blue</eyes>
</person>

For the sake of this question, pretend that person holds loads of other information. Pretend the file is like 10 MB.

Now, let's say you have another XML file which defines a group:

<group>
    <person>
        <name>John</name>
        <age>18</age>
        <hair>Brown</hair>
        <eyes>Blue</eyes>
    </person>
    <person>
        <name>Kim</name>
        <age>21</age>
        <hair>Blue</hair>
        <eyes>Green</eyes>
    </person>
    <person>
        <name>Sean</name>
        <age>22</age>
        <hair>Black</hair>
        <eyes>Brown</eyes>
    </person>
</group>

As you can see, if Person's were very large, the Group file would be extremely large. So, if we have something like John.xml, is there a standard way to reference it in Group.xml without explicitly defining all of John's data? I'm sure this is a very broad topic, so feel free to link me to any relevant web pages. Thanks!

Answer 1

There is no standard (will work in every parser) for importing nodes like that. But you could save space by changing some of your elements in to attributes

<group>
  <person name='John' age='18' hair='Brown' eyes='Blue' />
  <person name='Kim' age='21' hair='Blue' eyes='Green' />
  <person name='Sean' age='22' hair='Black' eyes='Brown' />
</group>

Answer 2

There are a couple of "standard" ways to do what you want, namely XLink and XInclude (depending on what you want to do), though you have to make sure that you have a processor that can pull in the external references. Most XML libraries don't come with this functionality already enabled.

Then you'd be able to do something like:

<group>
  <personlink xlink:href="person.xml" xlink:show="embed" xmlns:xlink="http://www.w3.org/1999/xlink"/>
</group>

However, you probably don't really need this. If you need a subset of information from a large document, you can easily use XSLT or XQuery to trim out the parts that you need. You can use this approach, along with SAX parsing - which is event based and doesn't have the whole document in memory - to scale you application to handle fairly large documents.

Even while using DOM, I didn't start to see problems with large documents until they were in the tens of megabytes range.

Answer 3

Um, there are no size limitations on xml files. you shouldn't worry about extremely large sizes. But remember; Xml is a data exchange format, not a database format. You use xml to swap data between different applications/services.

Answer 4

Standards

XInclude is the only standard with any level of support.

• Several XML editors, including Oxygen and xmlspy support it.
• Several XML parsers, including Xerces, also support it, and there are .net ports too.
• Several XML tools, such as Saxon support it, both for Java and .net.

There are some good examples of use in the Wikipedia article on XInclude.

XLink is a tangentially-related standard, not really for including documents, but more for citing portions within other documents. It's not well supported.

Alternatives

If you are worried about size, there are several ways to go:

• Use a streaming XML processor, such as DataDirect XQuery (or to a lesser extent, Saxon 9.3 EE, which only keeps as much information in memory as necessary to solve the query.
• Use an XML database, such as MarkLogic or eXist.
• Use one XML file to list the names of other XML files, which some program written in XQuery or XSLT then reads using the doc() function and processes. (Unless your processor is streaming or has a way to dispose of documents it is finished with, as DDXQ or Saxon do, you will still run into the same size problem through.)

Answer 5

Here is the XML specification for DTD, in which you can declare entity references.

A simple document like:

<!DOCTYPE test [
    <!ENTITY ref SYSTEM "file:///C:/test.txt" >
]>

<test>
    &ref;
</test>

And file:///C:/test.txt being:

<blah>
Fee
Fi
Fo
Fum
</blah>

will expand the original document to:

<test>
    <blah>
    Fee
    Fi
    Fo
    Fum
    </blah>
</test>

I do believe non-validating XML parsers are not required to expand out the references, so be cautious there.

Also, don't forget to put standalone="no" in the XMLDecl. (Not having the standalone attribute assumes it equals "no", but its still better to put it there...)