-->

How do you index attachment in Elasticsearch with

2019-04-14 20:54发布

问题:

Having difficulty indexing an attachment type in elasticsearch via the Tire gem. Not able to set attachment type correctly.

I've taken ActiveModel Integration example referenced from the Tire gem and added a field, filename, to reference name of PDFs on local filesystem that I want to index with the record.

#app/models/article.rb
class Article < ActiveRecord::Base
  include Tire::Model::Search
  include Tire::Model::Callbacks

  attr_accessible :title, :content, :published_on, :filename

  mapping do
    indexes :id, :type =>'integer'
    indexes :title
    indexes :content
    indexes :published_on, :type => 'date'
    indexes :attachment, :type => 'attachment' 
  end

  def to_indexed_json
    to_json(:methods => [:attachment])
  end

  def attachment
    if filename.present?
       path_to_pdf = "/Volumes/Disk41402/test_proj/sample_pdfs/#{filename}.pdf"
       Base64.encode64(open(path_to_pdf) { |pdf| pdf.read })
    end
  end
end

FWIW - the PDFs appear to have been added to the index:

$ curl -XGET 'http://localhost:9200/articles/_all/2?pretty=true'  
{
  "_index" : "articles",
  "_type" : "article",
  "_id" : "2",
  "_version" : 1,
  "exists" : true, "_source" : {"content":"Consectetur adipisicing elit, sed do eiusmod tempor incididunt. working?","created_at":"2012-06-21T17:19:03Z","filename":"sample2","id":2,"published_on":"2012-06-20","title":"Two","updated_at":"2012-06-28T00:00:59Z","attachment":"JVBERi0xLjMKJcTl8uXrp/Og0MTGCjQgMCBvYmoKPDwgL0xlbmd0aCA1IDAg\nUiAvRmlsdG...  ... ...4+CnN0YXJ0eHJlZgo4\nNTAzCiUVPRgo=\n"
}
}

But the attachment type is still "type" : "string" when it should be "type" : "attachment"

$ curl -XGET 'http://localhost:9200/articles/_mapping?pretty=true'
{
  "articles" : {
    "article" : {
      "properties" : {
        "attachment" : {
          "type" : "string"
        },
        "content" : {
          "type" : "string"
        },
        "created_at" : {
          "type" : "date",
          "format" : "dateOptionalTime"
        },
        "filename" : {
          "type" : "string"
        },
        "id" : {
          "type" : "long"
        },
        "published_on" : {
          "type" : "date",
          "format" : "dateOptionalTime"
        },
        "title" : {
          "type" : "string"
        },
        "updated_at" : {
          "type" : "date",
          "format" : "dateOptionalTime"
        }
      }
    }
  }
}

I've tried rebuilding the index rake environment tire:import CLASS=Article FORCE=true but type remains a string. Anyone see where I'm messing up?

Log (not exactly sure what 'no handler' means!?):

[2012-06-28 17:30:58,711][INFO ][cluster.metadata         ] [Kofi Whitemane] [articles] deleting index
[2012-06-28 17:30:58,765][WARN ][cluster.metadata         ] [Kofi Whitemane] [articles] failed to create
org.elasticsearch.index.mapper.MapperParsingException: mapping [article]
    at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:263)
    at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:211)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:680)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: No handler for type [attachment] declared on field [attachment]
    at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:259)
    at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parse(ObjectMapper.java:217)
    at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:161)
    at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:271)
    at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:174)
    at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:260)
    ... 4 more
[2012-06-28 17:30:58,805][INFO ][cluster.metadata         ] [Kofi Whitemane] [articles] creating index, cause [auto(bulk api)], shards [5]/[1], mappings []
[2012-06-28 17:30:58,891][INFO ][cluster.metadata         ] [Kofi Whitemane] [articles] update_mapping [article] (dynamic)

回答1:

Do you have the mapper attachment plugin installed?

See the tutorial at elasticsearch.org.