How to Implement Efficient Pagination in MongoDB Using _id and Forward Paging?

How to Implement Pagination in MongoDB

While using skip is generally discouraged for pagination due to its memory consumption at scale, an alternative approach is to leverage the _id field for natural ordering.

Getting the _id of the Last Document

To obtain the last document's _id on a given page, you can use the following approach:

<code class="javascript">db.users.find().limit(pageSize);
//Find the id of the last document in this page
last_id = ...;</code>

Forward Paging

Rather than using .skip() and .limit(), forward paging involves iterating through the cursor and storing the last seen _id. On subsequent iterations, the query is modified to exclude documents already seen, while also restricting results to those with a lower or equal _id to the last seen value.

For a natural order by _id, the code would look like this:

<code class="javascript">var lastSeen = null;
var cursor = db.junk.find().limit(3);

while (cursor.hasNext()) {
   var doc = cursor.next();
   if (!cursor.hasNext())
     lastSeen = doc._id;
}</code>

Modified Query for Subsequent Pages

The following query is used to retrieve the next page:

<code class="javascript">var cursor = db.junk.find({ "_id": { "$gt": lastSeen } }).limit(3);</code>

Complex Sorting

For sorting by fields other than _id, such as rank, you need to keep track of previously seen documents and use the $nin operator to exclude them.

Here's an example code snippet for forward paging with sorting by rank:

<code class="javascript">var lastSeen = null;
var seenIds = [];
var cursor = db.junk.find().sort({ "rank": -1 }).limit(2);

while (cursor.hasNext()) {
   var doc = cursor.next();
   if ( lastSeen != null && doc.rank != lastSeen )
       seenIds = [];
   seenIds.push(doc._id);
   if (!cursor.hasNext() || lastSeen == null)
     lastSeen = doc.rank;
}</code>

The subsequent query for the next page would be:

<code class="javascript">var cursor = db.junk.find(
    { "_id": { "$nin": seenIds }, "rank": "$lte": lastSeen }
).sort({ "rank": -1 }).limit(2);</code>

The above is the detailed content of How to Implement Efficient Pagination in MongoDB Using _id and Forward Paging?. For more information, please follow other related articles on the PHP Chinese website!