pd.DataFrame(page.data) on a page of OpenAPI Generator models returns seven columns named 0 to
6, and every cell is a pair such as ('id', 'ord_101'). On sdkgen entity objects it returns one
column of objects. Dump the models first and the status column prints OrderStatus.PAID in
every group-by. A loop that stops after the first page hands you three orders out of seven.
What you get
You will end up with one DataFrame built from every page of a list endpoint, with plain strings in its enum column and a row count checked against the objects read. This is for you if a vendor’s Python SDK hands you model objects rather than dictionaries.
Short answer
To turn typed SDK responses into a pandas DataFrame, read every page, convert each object to a
dict, and flatten the list with pd.json_normalize. On OpenAPI Generator’s Pydantic models, call
model_dump() and map the enum members with .value. On openapi-python-client models, call
to_dict(), which leaves UNSET fields out. sdkgen’s py-data package returns the frame from one
call. Then check that df.shape[0] equals the objects read.
You will need
Python 3.12 or later, pandas, and a Python SDK for the API installed from PyPI. Verified 2026-11-20
against Python 3.13.18, pandas 3.1.1, Pydantic 2.14.4, OpenAPI Generator 7.27.0,
openapi-python-client 0.30.0, and @voxgig/sdkgen 4.44.3. The document, api.json, describes an
order list behind a cursor envelope of data, next_cursor, and has_more, seven orders in
three pages. Every script runs against mock_api.py, a local stand-in that refuses a cursor it
did not issue.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| OpenAPI Generator python models with model_dump() | The SDK is OpenAPI Generator’s python client, with Pydantic models | A paging loop you write, Enum members to map, and float columns where integers go missing | Analysts want every page in one call with nullable integer columns |
openapi-python-client models with to_dict() | The SDK was generated by openapi-python-client and returns attrs models | A paging loop you write, dates as strings to parse, and UNSET wherever to_dict() is bypassed | You want Pydantic validation on every record |
| sdkgen py-data method | The API owner generates with sdkgen and publishes the py-data package beside the SDK | pandas only, a second package to install, and string columns for dates until you parse them | The vendor publishes no py-data package, or the analysts work in Polars |
| Speakeasy Python models with model_dump() | The vendor’s SDK is a Speakeasy Python SDK whose list operation has pagination configured | The same Enum and datetime handling as any Pydantic model, plus a next() loop per endpoint | The vendor’s document has no pagination extension, so next() has nothing to follow |
Voxgig maintains sdkgen. It is one of four options here, not the recommendation.
The two Pydantic rows keep the SDK’s validation until the last step, and a dict per object is the
price. The to_dict() that openapi-python-client generates writes strings for enums and dates,
so its frame needs one to_datetime and no enum mapping. The py-data package skips the object layer entirely, but it
exists only where the API owner generated and published it.
See what the DataFrame constructor makes of objects
The constructor treats a list of objects as a list of rows, and what a row turns into depends on the class.
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 one_column.py
sdkgen OrderEntity objects: shape (3, 1), columns [0], dtype object
OpenAPI Generator pydantic models: shape (3, 7), columns [0, 1, 2, 3, 4, 5, 6]
first cell: ('id', 'ord_101')
The sdkgen entity is opaque to pandas, so each row is one object. A Pydantic model is iterable,
and iterating it yields
(field_name, field_value) pairs, so pandas builds a column per field and stores the pairs.1
Group-bys and joins need a column per field, so each object has to become a dict first.
Read every page through the sdkgen SDK
generate.sh built all three clients once, from the same api.json, and the output is committed
under generated/. The sdkgen part is two commands: the scaffold adds the py target, the py-data
target that wraps it, and the paging feature.
(cd "$WORK" && npx --yes @voxgig/create-sdkgen@0.31.9 orderdesk --def api.json \
--target py --target py-data --feature paging)
(cd "$WORK/orderdesk-sdk/.sdk" && npm run generate)
The py SDK’s list() returns OrderEntity objects, and data_get() unwraps each one into a
dict. The paging feature
writes the server’s signals into the control dict you pass, so passing the same dict again
fetches the next page.
def sdkgen_records(sdk) -> list[dict]:
"""Every order through the sdkgen py SDK, unwrapped with data_get()."""
orders = sdk.Order()
ctrl = {"paging": {}}
records: list[dict] = []
seen: set[str] = set()
while True:
records.extend(order.data_get() for order in orders.list(None, ctrl))
if not ctrl["paging"].get("hasMore"):
return records
cursor = ctrl["paging"].get("cursor")
if cursor is not None and cursor in seen:
raise RuntimeError(f"pagination repeated cursor {cursor!r}")
seen.add(cursor)
sdkgen_frame.py runs that loop, then the py-data method, which drains the same pages and
returns the frame itself.
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 sdkgen_frame.py
py SDK, data_get() per entity: 7 objects, 7 rows, 3 requests
py-data, ad.orders(): 7 rows, 3 requests
GET /v1/orders?page=1
GET /v1/orders?cursor=cur_0003
GET /v1/orders?cursor=cur_0006
id status total items placed_at customer.country
0 ord_101 paid 42.5 3 2026-05-04T09:15:00Z NG
1 ord_102 shipped 18.0 1 2026-05-04T08:40:00Z NL
2 ord_103 pending 7.25 <NA> 2026-05-03T17:05:00Z TW
3 ord_104 refunded 99.0 4 2026-05-03T11:30:00Z IL
4 ord_105 paid 12.75 2 2026-05-02T20:00:00Z KE
5 ord_106 shipped 64.0 5 2026-05-02T10:10:00Z NG
6 ord_107 paid 30.0 2 2026-05-01T07:55:00Z IN
id string
items Int64
note string
placed_at string
status string
total Float64
customer.country str
customer.name str
placed_at with parse_dates: datetime64[us, UTC]
The first request carries page=1, because the feature sends a page number until a response hands
it a cursor; the stand-in ignores the parameter. The dtypes come from the
entity model, which records status as a $STRING and keeps none of its four enum values, so
this frame has no Enum members to map.
items stays an integer column with <NA> for the order that left it out, and placed_at stays
text until parse_dates names it.
Dump Pydantic models from OpenAPI Generator and Speakeasy
OpenAPI Generator’s python client has no paging helper; a search of the generated package for
paginat finds nothing. The loop is yours, and model_dump() turns each model into a dict.
def frame_from_pydantic(models: list) -> pd.DataFrame:
"""model_dump() keeps datetimes and Enum members; turn the members into strings."""
df = pd.json_normalize([m.model_dump() for m in models])
df["status"] = df["status"].map(lambda member: member.value)
return check_rows(df, len(models))
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 oag_frame.py
7 models from 3 requests, 7 rows
model_dump() status mapped
id str str
status str str
total float64 float64
items float64 float64
placed_at datetime64[us, UTC] datetime64[us, UTC]
note str str
customer.name str str
customer.country str str
status[0] after model_dump(): <OrderStatus.PAID: 'paid'>
status[0] after .value: 'paid'
placed_at[0]: 2026-05-04 09:15:00+00:00
model_dump(mode='json'): status str, placed_at str
The two dtypes columns are identical, which is the trap: the enum column reads str while it
holds OrderStatus members.2 placed_at arrived as datetime objects, and pandas 3.1 typed
them on its own, so no to_datetime call was needed here. model_dump(mode="json") gives plain
strings for both, which trades the enum fix for a date column you parse yourself.
json_normalize turned
the nested customer into customer.name and customer.country.
Speakeasy’s Python SDKs use Pydantic for every generated
model, so the same
frame_from_pydantic applies. Its paging is the x-speakeasy-pagination extension in the
vendor’s document, with type: cursor and an outputs.nextCursor path, and the response gains a
next() method that returns
nothing after the last page.3 Speakeasy was not run for this page, since generation needs an
account.
Check it worked
test_frames.py runs every route against the stand-in and compares each frame’s row count with the
objects iterated and the stand-in’s request count.
def test_the_pydantic_route_reads_every_page_and_maps_the_enum(self):
models = oag_models(self.oag())
df = frame_from_pydantic(models)
self.assertEqual((len(models), df.shape[0], len(self.api.seen)), (ORDER_COUNT, ORDER_COUNT, PAGES))
self.assertTrue(all(type(s) is str for s in df["status"]))
self.assertEqual(str(df["placed_at"].dtype), "datetime64[us, UTC]")
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 test_frames.py
test_a_cursor_that_repeats_stops_the_loop (__main__.Guards.test_a_cursor_that_repeats_stops_the_loop) ... ok
test_a_cursor_that_repeats_stops_the_sdkgen_loop (__main__.Guards.test_a_cursor_that_repeats_stops_the_sdkgen_loop) ... ok
test_check_rows_refuses_a_short_frame (__main__.Guards.test_check_rows_refuses_a_short_frame) ... ok
test_the_entity_model_keeps_status_as_a_plain_string (__main__.Guards.test_the_entity_model_keeps_status_as_a_plain_string) ... ok
test_attrs_asdict_leaks_unset_into_the_frame (__main__.Routes.test_attrs_asdict_leaks_unset_into_the_frame) ... ok
test_model_dump_leaves_enum_members_in_a_str_column (__main__.Routes.test_model_dump_leaves_enum_members_in_a_str_column) ... ok
test_py_data_returns_every_page_as_one_frame (__main__.Routes.test_py_data_returns_every_page_as_one_frame) ... ok
test_py_data_types_columns_from_the_model (__main__.Routes.test_py_data_types_columns_from_the_model) ... ok
test_the_attrs_route_reads_every_page_without_sentinels (__main__.Routes.test_the_attrs_route_reads_every_page_without_sentinels) ... ok
test_the_constructor_makes_one_object_column (__main__.Routes.test_the_constructor_makes_one_object_column) ... ok
test_the_pydantic_route_reads_every_page_and_maps_the_enum (__main__.Routes.test_the_pydantic_route_reads_every_page_and_maps_the_enum) ... ok
test_the_sdkgen_py_sdk_loop_reads_every_page (__main__.Routes.test_the_sdkgen_py_sdk_loop_reads_every_page) ... ok
test_the_stand_in_refuses_cursors_and_limits_it_did_not_document (__main__.Routes.test_the_stand_in_refuses_cursors_and_limits_it_did_not_document) ... ok
test_without_paging_py_data_stops_after_one_page (__main__.Routes.test_without_paging_py_data_stops_after_one_page) ... ok
Fourteen tests print ok and the runner ends with OK. check_rows raises ValueError when
df.shape[0] differs from the count, so a frame built from a short loop fails where it is made.
When it goes wrong
The group-by labels read OrderStatus.PAID in place of paid. model_dump() keeps
the Enum member, and printing one shows its class. Map the column with .value, as
frame_from_pydantic does.
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 pitfall.py
1. an enum column holds Enum members, and its dtype says str
dtype: str, type of a value: OrderStatus
status
OrderStatus.PAID 3
OrderStatus.PENDING 1
OrderStatus.REFUNDED 1
OrderStatus.SHIPPED 2
after .map(lambda member: member.value):
status
paid 3
pending 1
refunded 1
shipped 2
2. without the paging feature, the first page is the whole frame
rows: 3 of 7, requests: ['/v1/orders']
3. attrs.asdict() keeps the UNSET sentinel that to_dict() leaves out
attrs.asdict(): items object, row 3 items = Unset
to_dict(): items float64, row 3 items = nan
The frame holds three rows of seven. The client was constructed without feature={"paging": {"active": True}}, so py-data made one request and treated it as the only page. Pass the feature, and keep
the row-count check.
An items column turns to object and arithmetic on it raises. attrs.asdict() copied the
Unset sentinel that openapi-python-client puts in absent fields. Call the model’s own
to_dict(), which leaves them out, so pandas fills NaN.
When not to do this
Do not convert objects to a frame inside a service that only filters and forwards the records. The SDK’s types are the point there, and a frame throws them away for a table you never group.
Do not expect sdkgen’s py-data package from a vendor that ships only an OpenAPI Generator or
Speakeasy SDK. It exists when the API owner generates with sdkgen and publishes it; otherwise the
model_dump() loop here is the route.
Do not choose the py-data package for analysts who work in Polars. pandas is the only frame
library it knows. A Speakeasy SDK with pagination configured is the better source there, because
next() and model_dump() feed any frame library.
Do not trust dtypes to show an enum problem. The column reads str while it holds Enum
members, and only a value check, as in the test, finds them.
Related how-tos
Last verified
Verified 2026-11-20 against Node 22.23.5, Python 3.13.18, @voxgig/create-sdkgen 0.31.9,
@voxgig/sdkgen 4.44.3, @voxgig/apidef 8.32.4, OpenAPI Generator 7.27.0, openapi-python-client
0.30.0, pandas 3.1.1, Pydantic 2.14.4, and attrs 26.1.0. Every output block is what the command
preceding it printed. The three generators ran once, in generate.sh, and their Python packages
are committed under generated/. Speakeasy was not run. Its options come from its documentation.
Footnotes
-
Pydantic’s serialization page says so in one sentence, then adds that field values are left as is, so sub-models will not be converted to dictionaries. The same page documents
model_dump()two sections earlier. pandas, offered an iterable of pairs per row, takes each pair for a cell, which is a reasonable reading of an unreasonable input. ↩︎ Back to text -
The generated
OrderStatussubclassesstrandEnumtogether. Python 3.11 addedenum.StrEnum, whose__str__()isstr.__str__(), so its members print as their values. OpenAPI Generator’s python page gives the generator’s language version as 3.11+, and the template writes the mixin form that predates it. pandas 3.0 infersdtype="str"for a collection of strings, and the text guide says that without PyArrow it is backed by a NumPy object array. AnOrderStatusmember is astr, so it goes in as it came, class and all. ↩︎ Back to text -
Speakeasy’s pagination page shows the loop as
while response is not None, which is Python, and then says thenext()function returnsnil, nilwhen there are no more pages, which is Go. The page names no language and stands for every target that Speakeasy’s maturity table marks for pagination, so one sentence ended up in two. ↩︎ Back to text