How to crawl images in Nutch?

2019-07-21 15:43发布

问题:

How to crawl images in Nutch? Or, is there any other open search engine which is producing the results with images?

回答1:

change your regex-urlfilter.txt in conf

-.(ico|ICO|css|CSS|sit|SIT|eps|EPS|wmf|WMF|zip|ZIP|ppt|PPT|xls|XLS|gz|GZ|rpm|RPM|tgz|TGZ|exe|EXE|js|JS|gif|GIF|png|PNG||jpg|JPG|jpeg|JPEG|bmp|BMP|mpg|MPG|mov|MOV)$

Delete jpeg, jpg, gif or type picture that you want to grep.

And then change suffix-urlfilter.txt in conf

add # to jpeg, gif or png

That worked for me!



标签: solr nutch