How to index with ELKI - OPTICS clustering

Question

I'm an ELKI beginner, and I've been using it to cluster around 10K lat-lon points from a .csv file. Once I get my settings correct, I'd like to scale up to 1MM points.

I'm using the OPTICSXi algorithm with LngLatDistanceFunction

I keep reading about "enabling R*-tree index with STR bulk loading" in order to see vast improvements in performance. The tutorials haven't helped me much.

Any tips on how I can implement this feature?

Answer 1

The suggested parameters for using a spatial R* index on 2 dimensional data are:

-db.index tree.spatial.rstarvariants.rstar.RStarTreeFactory
-pagefile.pagesize 512
-spatial.bulkstrategy SortTileRecursiveBulkSplit

For higher dimensional data, larger page sizes are necessary. A page size of 512-1024 bytes seems to be the sweet spot for 2 dimensional data, but it does depend on your data, too.

To discretize clusters, you can use the Xi extraction:

-algorithm clustering.optics.OPTICSXi -opticsxi.xi 0.005

To benefit from index acceleration with OPTICS, choose epsilon as small as possible for your application. The parameter is in meters with all the earth models in ELKI.

-opticsxi.algorithm OPTICSHeap
-algorithm.distancefunction geo.LatLngDistanceFunction
-optics.epsilon 2000.0 -optics.minpts 10

uses 2 km distances maximum.

Make sure to distinguish latitude,longitude and longitude,latitude. Both orders are used, and you need to use the right distance function:

geo.LatLngDistanceFunction
geo.LngLatDistanceFunction