Adding child documents to existing Solr 6.4 collec

Question

This question is similar to Solr doesn't overwrite - duplicated uniqueKey entries, but I am in a situation where I have a large body of existing documents that have already been added to the collection with no child documents, and I am using (standalone not cloud) Solr 6.4 rather than 5.3.1. We recently enabled child documents so that we could store richer data.

We use SolrJ to load data into and query Solr, but to isolate the issue we're seeing, I used the command line Solr post tool to upload the following document:

<add>
    <doc>
        <field name="id">1</field>
        <field name="solr_record_type">1</field>
        <field name="title">Fabulous Book</field>
        <field name="author">Angelo Author</field>
    </doc>
</add>

Search results were as expected: Using q=id:1 and fl=id,title,index_date,[child parentFilter="solr_record_type:1"]

 "response":{"numFound":1,"start":0,"docs":[
      {
        "id":"1",
        "title":"Fabulous Book",
        "index_date":"2019-01-16T23:06:57.221Z"}]
  }

Then I updated the document by posting the following:

<add>
    <doc>
        <field name="id">1</field>
        <field name="solr_record_type">1</field>
        <field name="title">Fabulous Book</field>
        <field name="author">Angelo Author</field>
        <doc>
            <field name="id">1-1</field>
            <field name="solr_record_type">2</field>
            <field name="contributor_name">Polly Math</field>
            <field name="contributor_type">3</field>
        </doc>
    </doc>
</add>

Then, repeating my search, I got the following duplicate result, searching on the unique id field, which is undesirable.

    "response":{"numFound":2,"start":0,"docs":[
      {
        "id":"1",
        "title":"Fabulous Book",
        "index_date":"2019-01-16T23:06:57.221Z",
        "_childDocuments_":[
        {
          "id":"1-1",
          "solr_record_type":2,
          "contributor_name":"Polly Math",
          "contributor_type":3,
          "index_date":"2019-01-16T23:09:29.142Z"}]},
      {
        "id":"1",
        "title":"Fabulous Book",
        "index_date":"2019-01-16T23:09:29.142Z",
        "_childDocuments_":[
        {
          "id":"1-1",
          "solr_record_type":2,
          "contributor_name":"Polly Math",
          "contributor_type":3,
          "index_date":"2019-01-16T23:09:29.142Z"}]}]
  }

Going the other way, if I start with a document that was loaded initially with a child document, like the following:

<add>
    <doc>
        <field name="id">2</field>
        <field name="solr_record_type">1</field>
        <field name="title">Wonderful Book</field>
        <field name="author">Andy Author</field>
        <doc>
            <field name="id">2-1</field>
            <field name="solr_record_type">2</field>
            <field name="contributor_name">Polly Math</field>
            <field name="contributor_type">3</field>
        </doc>
    </doc>
</add>

And then I update it with a document with no children:

<add>
    <doc>
        <field name="id">2</field>
        <field name="solr_record_type">1</field>
        <field name="title">Wonderful Book</field>
        <field name="author">Andy Author</field>
    </doc>
</add>

The result still has the child:

  "response":{"numFound":1,"start":0,"docs":[
      {
        "id":"2",
        "title":"Wonderful Book",
        "index_date":"2019-01-16T23:09:39.389Z",
        "_childDocuments_":[
        {
          "id":"2-1",
          "title_id":2,
          "title_instance_id":2,
          "solr_record_type":2,
          "contributor_name":"Polly Math",
          "contributor_type":3,
          "index_date":"2019-01-16T23:07:04.861Z"}]}]
  }

This is strange because if I update a document with 2 child documents with a replacement document with only 1 child document, it does drop one child document. But in this case, it is not dropping the child document.

Updates of documents with no child documents that don't add child documents, and updates of documents with child documents that don't remove all child documents both seem to work as I'd expect.

I have a large body of existing documents that don't have children, which I may be adding children to, and eventually I may have a lot of child-having documents that might drop their children. Given that, what is the best way to update these records without generating duplicate records or losing updates?

Answer 1

I would strongly advise avoiding Solr parent/child relationships. We decided to use them in Solr 5.3.1 and it turns out that although much of the functionality is there, there are a number of nasty bugs present in Solr since 4.x that remain unfixed including

• SOLR-6096: Support Update and Delete on nested documents
• SOLR-5211: updating parent as childless makes old children orphans (UPDATE: fixed in 8.0)
• SOLR-6596: Atomic update and adding child doc not working together
• SOLR-5772: duplicate documents between solr "block join" documents and "normal" document
• SOLR-10030: SolrClient.getById() method in Solrj doesn't retrieve child documents

For those reasons, if at all possible, I strongly recommend AVOID using child documents. Even if those issues don't hit you now they will in the future at some point and it's clear, given that they have not been fixed in 3 to 4 major versions, that there is no real support in the product for child documents. Sorry to be the bearer of bad news but hopefully someone can learn from our experience.