pagination is common design pattern that is used everywhere across the web. It is the last thing you see on search results page.

Lets say If we want to get the list of tweets from twitter, there could be thousands of them. So the logical way is to limit the results to 50 and provide a way to fetch others(discussed later).

But there is a small catch, it is not guaranteed that you'll see all the results(with very few exceptions).

1. Let's take an example, you've searched for something on google and on bad day you don't see any relevant results on the first 10 links shown.
2. While you are on the first page google has decided that 11'th result (first one in the second page) is getting more hits and decided to exchange places with the last one in page 1
3. And now once we click on the page 2 we see a result that we've have already seen on page 1, to make things worse we might never see the 11th result ever.

This is fair trade-off when Users are consuming the data visually given how rarely the results get updated.

Most of the public API's follow similar pattern of limiting the results and providing a mechanism to fetch more.

One common pattern used is Continuation tokens, In this pattern a token is added to the response of the request ,that has reference to next page(n+1). Next set of results can be fetched using the token from that is received as response from the previous page. Here is the sample API response

results:[1-50],
continuationToken: "3593-some-random-number" // (use this to get results from 51-100)
}

All is fine until we try to do some sort of aggregation on the results from paginated API's. Imagine trying to find out the mean of the scores from a University API and all the inconsistencies we end up with. if iterated sequentially orange is read twice and banana is never read

It might look like ordering of results by timestamp(if the API supports) might solve the problem, but this fails when the records can be removed and the results might move up in the order.

Conclusion: No way of getting paginated data is perfect. Making sure of this limitation while in designing is the only way. Thanks for reading :)