Security implications of letting users render own

I plan to let website users upload their own SVG documents and render them with inkscape or svg2pdf. The users will either be unauthenticated or go through a trivial sign-up process, so I would expect some hack attempts. I should therefore appreciate any pointers on what filtering I can do to minimise security threats.

Inkscape doesn't seem to be bothered by JavaScript onload tags and happily renders the content without anything untoward happening (that said, I can't get Firefox 10 to cough up an alert box either using that approach).
I am concerned that an <image xlink:href /> tag could link to a huge or malformed bitmap image using an external URI - which theoretically could crash the service. Is there an easy way to traverse the XML document to filter these? I can do this easily with XMLReader of course, but wonder if I might have to deal with things like onload for 'onload' (though Firefox just rejected it as invalid, so perhaps this is a needless worry). Sidenode: images in themselves are acceptable but I think I'd either require them to either be inline data: or whitelist acceptable target URIs, with filesize limitations.
Are there any SVG directives (in particular that render text) that could include the text contents of system files, such as /etc/passwd etc?
One approach I could also take is validation against the SVG spec. That's the subject of another question I've asked here.

I'm using PHP 5.2 with XMLReader and XMLWriter, though other PHP stream-based systems would be acceptable. Systems are OS X 10.6.8 for dev, and LAMP on production.