Custom Spotlight Importer and Finder's Get Inf

Question

I have written a Spotlight Importer for the custom document type my application defines.

Everything is working fine, the metadata fields are correctly indexed by Spotlight (verified using mdls command), and a Spotlight search reveals my documents.

The only problem I have is that items I specify in the <displayattrs> section of the schema.xml file aren't displayed in the "More Info" section when I ask for informations about a file (Cmd+I in the Finder).

I expected these fields to appear there because I declared them both in the <allattrs> and <displayattrs> sections.

I found few questions here related to this problem, none of them helped me.

The importer is bundled into the app, loaded by the system (mdimport -L confirmed this). Also, the bundle structure seems right, the schema.xml appears in the Resources folder, as well as the schema.strings in the en/lproj folder.

Here is what the schema.xml file look like :

<schema version="1.0"
    xmlns="http://www.apple.com/metadata"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.apple.com/metadata file:///System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Resources/MetadataSchema.xsd">
<types>
    <type name="com.mydomain.myapp.mydocument">
        <allattrs>
            kMDItemTitle kMDItemAuthors kMDItemAlbum
        </allattrs>
        <displayattrs>
            kMDItemTitle kMDItemAuthors kMDItemAlbum
        </displayattrs>
    </type>
</types>

A couple of things more, my system is lacking the mdcheckschema command, but the XML file is so short, I doubt there is a problem with the syntax.
Sometimes, the "More Info" section display the file last opening date, sometimes nothing.
Finally, I tried reimporting the file (mdimport), to no avail.

I'm running Mac OS X Moutain Lion 10.8.3, Xcode 4.6.2.

So here my question, am I missing something to have those items displayed in the "More Info" section ? Is there someone who has experienced such a problem and found a solution ?

Edit :

Nobody answered my question so far, maybe someone can point me to some tutorial or documentation about this problem ?

Answer 1

I know that Vince has probably long since solved this (or given up). But I've just spent a frustratingly long time working through various poorly documented or totally undocumented issues with writing an importer, so I thought I'd document my findings here. (I'm afraid this has turned into an essay - it's a complicated subject).

Let's assume:

• You've read the documentation on how to write a Spotlight importer, in particular the troubleshooting guide.

• You've written and debugged your importer.

  To debug your importer in Xcode choose Product->Scheme->Edit Scheme and set:

  • Info->Executable to /usr/bin/mdimport
  • Arguments->Arguments to -n -d2 -g $(BUILT_PRODUCTS_DIR)/$(WRAPPER_NAME) /path/to/some/test/file.ext
  • Options->Working Directory to $(SRCROOT)

  And set a breakpoint on your GetMetadataForURL() function.

• The output from /usr/bin/mdimport -n -d2 -g /path/to/your/importer.mdimporter /path/to/some/test/file.ext correctly contains the standard and/or custom metadata attributes that you intended.
• You've deployed your importer for testing (either standalone in /Library/Spotlight/ or embedded in an app bundle) and mdimport -L lists your importer.
• But the output of mdls /some/other/file.ext and/or Finder's "Get Info" window doesn't show the metadata attributes that you expected.

Here's some things to check:

1. Someone else needs to declare the UTI(s) for the document type(s) that you're importing.

   • If you're importing a document of a system-declared type then OSX has declared the UTI for you.
   • If your importer is embedded in an app bundle, then the app should declare the UTI via a UTExportedTypeDeclarations key in the app's Info.plist.
   • If you're importing a third-party document type then check that the app that "owns" the document type has declared a UTI for it in a UTExportedTypeDeclarations key in the app's Info.plist. If the app hasn't declared a UTI (some don't and still use the old CFBundleDocumentTypes->CFBundleTypeExtensions key instead) or if you want your importer to work even if the app isn't installed then you will have to create a "dummy" app whose sole purpose is to declare the UTI(s) in a UTImportedTypeDeclarations key in the app's Info.plist. Install the "dummy" app somewhere like /Library/Application Support/myOrg/myApp.app. Your importer should be standalone and should not be embedded in this app's bundle since Spotlight won't run importers from an app that the user hasn't opened.

   There's no point declaring the UTI(s) that you're importing in UTImportedTypeDeclarations or UTExportedTypeDeclarations keys in your importer's Info.plist - LaunchServices won't reliably read them from there so Spotlight won't recognise them. However you must register your interest in the UTI(s) by refering to them in CFBundleDocumentTypes->LSItemContentTypes key(s) in your importer's Info.plist.

   Symptoms of someone else not having correctly declared a UTI are that mdimport -n -d1 /some/file.ext says:

   • Imported '/some/file.ext' of type 'dyn.xxx' ... or (confusingly):
   • Imported '/some/file.ext' of type 'the.correct.uti' with no plugIn.

   .

2. If an attribute that your importer returns is not listed in the metadata schema for your document's UTI, or for any parent UTIs, then Spotlight throws that attribute away. Even if it's a standard attribute like kMDItemAuthors. To understand why, we need to look at how Spotlight works in detail:

   • An app declares one or more UTIs in a UTImportedTypeDeclarations or UTExportedTypeDeclarations key.

   • In each UTI declaration, the app specifies one or more 'parent' UTIs in a UTTypeConformsTo key. The parent UTI should be something specific if possible - e.g. "public.image" if the app is declaring a new type of image file - or just "public.data" if nothing else is appropriate.

     • You can see the current state of the UTI hierarchy by poring over the contents of the LaunchServices database: /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -dump .
     • But that's tricky to decipher. Fortunately you'll normally be more interested in the UTI hierarchy of a 'clean' machine which can be obtained by plutil -p /System/Library/CoreServices/CoreTypes.bundle/Contents/Info.plist .

   • Spotlight maintains a "schema" which lists the metatdata attributes of interest to it:

     • You can see the current state of the metadata schema with mdimport -X 2>&1 .
     • You can see the the metadata schema of a 'clean' machine in /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Resources/schema.plist .

   • When Spotlight is deciding what to store it cross-references the output of your importer against both the UTI hierarchy and the metadata schema. So for each attribute that your importer returns:

     • Spotlight looks up the document's UTI in the metadata schema. If there exists an entry for the UTI then Spotlight checks whether the attribute that your importer returns is listed under the allattrs key. If it is then Spotlight records the value provided by your importer in its database.
     • Otherwise Spotlight looks up the parent UTI in the UTI hierarchy and repeats the process until it hits "public.data".
     • If Spotlight can't find the attribute listed in the allattrs key for your document's UTI, or for any parent UTIs, then it throws away the value provided by your importer.

   .

3. If an attribute that is stored in the Spotlight database is not listed for display in the metadata schema for your document's UTI, or for any parent UTIs, then Finder's "Get Info" window won't display it. Even if it's a standard attribute like kMDItemAuthors.

   • Finder follows a similar process to Spotlight above, except that it consults the displayattrs keys instead of the allattrs keys in the metadata database.
   • The order in which attributes are displayed depends on their position in the metadata schema hierarchy.

   .

4. If you want to control what Spotlight stores and/or what Finder's "Get Info" window displays then your importer needs to supply a custom schema.

   • The format for the custom schema.xml is fairly well documented. Unfortunately the mdcheckschema command mentioned in the documentation no longer ships with Xcode. If you have a machine with an older version of OSX & Xcode you can copy it from /usr/bin/mdcheckschema. If you have an Apple Developer account you can extract it from /Packages/DeveloperToolsCLI.pkg on the "Xcode 4.2 for Snow Leopard" dmg.
   • You don't have to list every attribute that your importer supports in the allattrs and displayattrs keys - only those attributes that aren't listed for a parent or grandparent UTI in /System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Resources/schema.plist .
   • However if you want to control the order in which attributes are displayed in the "Get Info" window you should list the attributes that you want displayed first in your desired order in the displayattrs key. (See for example "public.movie" in the schema, which duplicates some keys from its parent "public.audiovisual-content" so that they're displayed first).
   • Your schema must define at least one custom attribute in the attributes section and reference to it in an allattrs key, otherwise Spotlight ignores the whole schema. If your importer doesn't supply any custom attributes then just add a bogus custom attribute to the schema anyway. (This requirement arrived some time after Snow Leopard and is completely undocumented, and is probably where Vince was going wrong).
   • If your schema defines a custom attribute (and it should; see previous point) then you must supply an English schema.strings localization for it, otherwise Spotlight ignores the whole schema. (Of course you're welcome to provide other localizations too).
   • Check that you have a "Copy Bundle Resources" phase in your Xcode project that copies schema.xml and schema.strings into your product.
   • Double-check that Contents/Resources/schema.xml and Contents/Resources/en.lproj/schema.strings or Contents/Resources/English.lproj/schema.strings really exist in your built product; some older versions of Xcode didn't copy them across.
   • Check that file /path/to/your/built/importer.mdimporter/Contents/Resources/en.lproj/schema.strings says:
     • Little-endian UTF-16 Unicode c program text .

   A symptom of getting any of the above wrong is that mdimport -X 2>&1 | grep -A20 uti.of.interest either returns nothing or returns an empty schema for the UTI that your importer's schema.xml is trying to define.

5. Spotlight doesn't always notice changes in a timely manner.

   • When testing an updated version of your importer first delete the old importer (or the entire app that contains it if it's embedded in an app bundle) and type mdimport -L to check that Spotlight has noticed that it's gone (this might take ~30s) before deploying your updated version. Type mdimport -L again to check that Spotlight has noticed the updated version (again this might take ~30s) before resuming testing.

   • If you're distributing a standalone importer in a .pkg file, then you should include a postinstall script to 1: tell LaunchServices that the bundle has been updated (Installer does this automatically for apps, but not for other bundle types) and 2: kick Spotlight into re-indexing for the current user the document types that your importer understands:

     #!/bin/sh touch -c "$2" if [ -n "$USER" ]; then sudo -u "$USER" /usr/bin/mdimport -r "$2"; fi true

6. LaunchServices doesn't always notice changes in a timely manner, and keeps old information lying around.

   • If you're making changes to the declaration of the UTI(s) in the app that declares them, or to the UTI(s) that your importer registers for then LaunchServices and Spotlight can get confused. You can completely reset LaunchServices and get it to re-read from standard places with:

     /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -v -kill -seed -domain system -domain network -domain local -domain user

     This is also useful if you want to simulate a 'clean' install of your importer and/or app on your development system.

Edit: This project on gitHub illustrates points 1-5 above.