How to Switch from Offset to Cursor Pagination

Modern Treasury’s API offers the ability to list through resources such as Payment Orders, Counterparties, External Accounts, Transactions, and more through List endpoints. For performance reasons, the API will return only a subset of the entire result set in each API call.

To obtain the entire result set for a specific type of resource, like all Payment Orders, the API accepts optional query parameters to progress to the next subset of results. For example, if there are 50 total Payment Orders to obtain, the first API request to the List endpoint can retrieve the first 25 Payment Orders, and the next API request can retrieve the remaining 25. This process is called “pagination.” Modern Treasury is shifting from offset, or page-based, pagination to cursor-based pagination to offer improved accuracy and efficiency. This article will help with the transition from the deprecated offset style pagination to the improved cursor pagination.

Using Cursor Pagination

As a result of moving to cursor-based pagination, the headers in the API response to a List endpoint will no longer contain the keys X-Total-Count and X-Page and instead will introduce an X-After-Cursor header. The page query parameter will be deprecated and functionally replaced with the after_cursor query parameter.

To transition from offset pagination to cursor pagination, you have two options:

1. If your integration is in Python or Javascript, you can use Modern Treasury's beta API Libraries & SDKs. These API Libraries support cursor pagination out-of-the-box, and offer auto-pagination which eliminates the need to fetch pages manually. Further documentation can be found on that page.
2. Remove the page query parameter in your List requests. Instead, use the value of the the X-After-Cursor header in the after_cursor parameter of the subsequent request to obtain the next set of results. If the X-After-Cursor header contains an empty value, that means there are no more results to be returned. Note that the last page of results will still contain an X-After-Cursor which is the ID of the final result. When polling, you will have to fetch one more time and get an empty JSON array to get the empty X-After-Cursor.

Note that cursor pagination still supports the per_page parameter.

For example:

A GET request to our payment orders endpoint with URL https://app.moderntreasury.com/api/payment_orders?per_page=25

will return a response body with the first 25 payment orders and will transmit response headers that will contain the following:

Example Response Headers

The value for X-After-Cursor may be used to retrieve the next set of results. The size of the results are determined by the per_page parameter. For example, to retrieve the next 25 results, the request URL should be https://app.moderntreasury.com/api/payment_orders?per_page=25&after_cursor=<cursor_id>. Again, if the X-After-Cursor header, <cursor_id>, is null, then you may conclude pagination as there are no more results to retrieve.

Filter parameters

Filtering on our List endpoints is still supported. Any filter query parameter may be appended to the URL and used in conjunction with cursor pagination in the same manner as with offset pagination.

per_page parameter

The per_page parameter is unchanged and allows you to specify how many records to fetch per request, up to a maximum of 100. The default value is still 25.

Jumping and Counts

Offset pagination offers the ability to jump to an arbitrary page number while skipping all the pages preceding the selected page. Unfortunately, this ability is lost with cursor pagination, since you must traverse each subset of results to obtain the next cursor needed for the subsequent request.

For performance reasons, the total count of the result set, represented by the X-Total-Count header, will no longer be returned when using cursor pagination.

Python Example

import requests

org_id = "your organization id"
api_key = "your api key"

url = 'https://app.moderntreasury.com/api/payment_orders?per_page=25'
response = requests.get(url, auth=(org_id, api_key))
data = response.json() # read the payment orders result set
cursor = response.headers['X-After-Cursor']
while cursor:
	response = requests.get(f"https://app.moderntreasury.com/api/payment_orders?per_page=25&after_cursor={cursor}", auth=(org_id, api_key))
	data = response.json() # read the payment orders result set
	cursor = response.headers['X-After-Cursor']

Ruby Example

require "uri"
require "net/http"
require "openssl"
require "json"

BASE_URL = "https://app.moderntreasury.com/api/"
ORG_ID = "your organization id"
API_KEY = "your api key"

def get(path)
  uri = URI(BASE_URL + path)

  Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |http|
    request = Net::HTTP::Get.new(uri)
    request["Accept"] = "application/json"
    request["Content-Type"] = "application/json"
    request["Authorization"] = request.basic_auth(ORG_ID, API_KEY)
    http.request(request)
  end
end

response = get("payment_orders?per_page=25")
data = JSON.parse(response.body) # read the payment orders result set
cursor = response["X-After-Cursor"]
until cursor.nil?
  response = get("payment_orders?per_page=25&after_cursor=#{cursor}")
	data = JSON.parse(response.body) # read the payment orders result set
  cursor = response["X-After-Cursor"]
end

Communication Timeline