paginated data with the help of mongo inbound adap

Question

I am using mongo inbound adapter for retrieving data from mongo. Currently I am using below configuration.

<int-mongo:inbound-channel-adapter
        id="mongoInboundAdapter" collection-name="updates_IPMS_PRICING"
        mongo-template="mongoTemplatePublisher" channel="ipmsPricingUpdateChannelSplitter"
        query="{'flagged' : false}" entity-class="com.snapdeal.coms.publisher.bean.PublisherVendorProductUpdate">
        <poller max-messages-per-poll="2" fixed-rate="10000"></poller>
    </int-mongo:inbound-channel-adapter>

I have around 20 records in my data base which qualifies the mentioned query but as I am giving max-messages-per-poll value 2 I was expecting that i will get maximum 2 records per poll. but I am getting all the records which qualifies the mentioned query. Not sure what I am doing wrong.

Answer 1

Actually I'd suggest to raise a New Feature JIRA ticket for that query-expression to allow to specify org.springframework.data.mongodb.core.query.Query builder, which has skip() and limit() options and from there your issue can be fixed like:

<int-mongo:inbound-channel-adapter
    query-expression="new BasicQuery('{\'flagged\' : false}').limit(2)"/>

Answer 2

The mongo adapter is designed to return a single message containing a collection of query results per poll. So max-messages-per-poll makes no difference here.

max-messages-per-poll is used to short-circuit the poller and, in your case, the second poll is done immediately rather than waiting 10 seconds again. After 2 polls, we wait again.

In order to implement paging, you will need to use a query-expression instead of query and maintain some state somewhere that can be included in the query on each poll.

For example, if the documents have some value that increments you can store off that value in a bean and use the value in the next poll to get the next one.