Skip to main content

Pagination

Endpoints that return large amounts of data may support pagination to improve the performance of the API and provide a better user experience. Pagination is the process of dividing large collections of resources into smaller, more manageable chunks, or 'pages'. By limiting the number of resources returned in each page, the API can reduce the amount of data that needs to be transmitted over the network, which can improve performance and reduce latency. For example, the endpoint GET v3/shipments would return a large number of shipment resources if pagination was not implemented. A single response could easily consist of thousands of shipments, which could take a long time to generate and transmit over the network. This could result in a poor user experience, with slow load times and increased network usage. With pagination, the API can divide the records into smaller chunks, such as 100 records per page, and return only the requested page to the user. This reduces the amount of data transmitted over the network and can improve performance and response times. Pagination also provides additional benefits, such as the ability to navigate large datasets and reduce the risk of hitting resource limits, such as memory or processing power, on the server or client side.

There are two flavours of pagination available:

  • Cursor-based pagination
    This uses the query parameters ?page[cursor]={cursor} and page[size]={size}.
  • Number-based pagination (deprecated)
    This uses the query parameters ?page[page]={page} and page[size]={size}.

The two cannot be used simultaneously or interchangeably.

Cursor-based pagination​

Cursor-based pagination is a pagination technique that uses a cursor to mark a specific point in a collection, and returns a portion of the collection that comes after that point.

The cursor is typically a unique value that helps the server identify the current,next or previous page of the dataset being paginated. Note that is must be considered as abstract and meaningless data, even though it may be recognizable as id, timestamp or some other form of information. Cursor-based pagination can be more efficient than traditional number-based pagination because it avoids the need to skip a large number of resources in the collection. With traditional page-based pagination, each page request requires the server to retrieve the entire set of resources up to the page being requested, and then skip over the resources that have already been returned in previous pages. In contrast, with cursor-based pagination, the server only needs to retrieve the resources that come after the cursor, and return them to the client. This can be particularly useful when working with large collections or real-time data sources, where the data may be constantly changing and the client needs to efficiently retrieve the latest updates. An advantage of cursor-based pagination is that it is less prone to 'page-shifting' where resources on a particula page shift from one to the next because the underlying collection changed. When an endpoint supports cursor-based pagination it would return the following links in accrodance to the JSON:API specification for pagination.

For example, GET {host}/v3/shipments?page[size]=100 retrieves the first page of shipment resources with a maximum of 100 items per page. The response includes pagination links that allow a client to navigate to the next, previous, first, and last pages of the result set as follows:

{
    "data": [
      /** all items go here **/
    ],
    "links": {
    "first": "https://{host}/v3/shipments?page[cursor]=0&page[size]=100",
    "self": "https://{host}/v3/shipments?page[cursor]=0&page[size]=100",
    "next": "https://{host}/v3/shipments?page[cursor]=68076953&page[size]=100"
    }
}

When invoking the next link the next 100 results will be fetched from the server and this process can be repeated until the response contains fewer items than requested using the page[size] query parameter or when no next link is provided. For example the following response will be of an empty result:

{
    "data": [
      /** empty **/
    ],
    "links": {
    "first": "https://{host}/v3/shipments?page[cursor]=0&page[size]=100",
    "self": "https://{host}/v3/shipments?page[cursor]=95509099&page[size]=100",
    "next": null /** the node might be omitted as well **/
    }
}

Depending on the implemntation of the endpoint it may be possible to periodically repeat the request to the last 'next' link to determine if there are newer results. If there are then the page will contain items and a new next link will be returned, otherwise the result will remain an empty page. Beware to not spam the endpoint(s) and prevent excessive polling for new data. This behaviour may cause all future requests to be blocked.

Number-based pagination (deprecated)​

Number-based pagination is a pagination technique that uses page numbers and page sizes to divide a large data set into smaller, more manageable pages. With number-based pagination, the API client specifies the page number and page size it wants to retrieve, and the server returns a specific "page" of results that corresponds to that page number and page size. For example, if the client requests page 2 with a page size of 10, the server returns the 11th through 20th items in the data set. Number-based pagination can be simple to implement and understand, but it has the disadvantage that it is less efficient than cursor-based pagination when working with very large data sets, because the server may need to retrieve and count a large number of resources in order to jump to the requested page. Additionally, changes to the collection (such as adding or deleting resources) can cause the page numbers to change, which is referred to as 'page-shifting'. Number-based pagination is not supported by all endpoints due to this page-shifting and the fact that it is more resource intensive. For example, GET {host}/v3/shipments?page[number]=2&page[size]=100 retrieves the second page of shipments with a maximum of 100 items per page. The response includes pagination links that allow a client to navigate to the next, previous, first, and last pages of the result set as follows:

{
    "data": [
    /** all items go here **/
    ],
    "links": {
    "first": "https://{host}/v3/shipments?page[number]=1&page[size]=100",
    "self": "https://{host}/v3/shipments?page[number]=2&page[size]=100",
    "next": "https://{host}/v3/shipments?page[number]=3&page[size]=100"
    }
}