Elastic search application side join - pagination

问题:

I have many-to-many relationship in db. when records are indexed into Elastic search will be using application side join. given joining two resultsets from Elasticsearch will have to be merged at an application level - will be losing out of the box pagination that Elastic search offers (on merged results sets at application side) and also most importantly aggregations. trying to find out way to go about leveraging ES aggregations and pagination while using application side join and merging results on application side?