APIs and Async · Building an API Client · lesson 11 of 12
Pagination
about 16 minutes · free · runs in your browser
Collections that do not fit in one response
An API that returns thousands of rows sends them a page at a time:
{"page": 1, "total_pages": 3, "has_next": true, "users": [...]}
Fetching them all means looping until the server says stop. Two rules keep it safe:
- Trust the server's signal, not your own arithmetic.
has_nextor comparing againsttotal_pagesbeats guessing when an empty page means the end. - Have a ceiling. A bug in the API — or in your loop — should not fetch a million
pages. A
max_pageslimit turns a runaway into a bounded mistake.
The fake /users endpoint returns two users per page across three pages.
Your turn: write fetch_all_users(max_pages=10) returning every user, following the
pages until has_next is false.
You start from this, and edit it in the browser:
import fake_requests as requests
URL = "https://api.pylearn.dev/users"
def fetch_all_users(max_pages=10):
# Follow the pages and return every user.
pass