Sentry Answers>FastAPI>

FastAPI and MongoDB: ObjectId object is not iterable error

FastAPI and MongoDB: ObjectId object is not iterable error

David Y.

—

The Problem

I’m building a calendar application with FastAPI, using MongoDB for data storage. The endpoint I’ve written for event creation takes data based on an Event model I’ve defined and should store that data in the MongoDB database and then return it to the user. However, at the moment, calls to this endpoint fail with the following errors:

Click to Copy
ValueError: [TypeError("'ObjectId' object is not iterable"), TypeError('vars() argument must have __dict__ attribute')]

Here’s my application code:

Click to Copy
from fastapi import FastAPI
from pymongo import MongoClient
from pydantic import BaseModel
from datetime import datetime

app = FastAPI()

class Event(BaseModel):
    title: str
    description: str
    start_time: datetime
    end_time: datetime

def store_in_mongo(collection: str, event: dict):
    client = MongoClient("mongodb://localhost:27017")
    db = client["calendar_db"]
    collection = db[collection]
    result = collection.insert_one(event)
    return result.acknowledged

@app.post("/events/")
async def create_event(event: Event):
    event_dict = event.dict()
    result = store_in_mongo("events", event_dict)

    if result:
        return event_dict
    else:
        return {"message": "Failed to create event."}

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="0.0.0.0", port=8000)

Why am I getting these errors and how can I fix them?

The Solution

This error occurs because PyMongo’s insert_one method mutates event_dict by adding an '_id' key with a MongoDB ObjectID object as a value. We can see this in action by adding print statements to create_event before and after store_in_mongo is called:

Click to Copy
@app.post("/events/")
async def create_event(event: Event):
    event_dict = event.dict()
    print(event_dict)
    result = store_in_mongo("events", event_dict)
    print(event_dict)

    if result:
        return event_dict
    else:
        return {"message": "Failed to create event."}

If we call the endpoint now, we will see lines similar to the following in our console output:

Click to Copy
{'title': 'Sunday Lunch', 'description': 'Lunch with family.', 'start_time': datetime.datetime(2024, 6, 30, 8, 25, 19, 294000, tzinfo=TzInfo(UTC)), 'end_time': datetime.datetime(2024, 6, 30, 14, 25, 19, 294000, tzinfo=TzInfo(UTC))}
{'title': 'Sunday Lunch', 'description': 'Lunch with family.', 'start_time': datetime.datetime(2024, 6, 30, 8, 25, 19, 294000, tzinfo=TzInfo(UTC)), 'end_time': datetime.datetime(2024, 6, 30, 14, 25, 19, 294000, tzinfo=TzInfo(UTC)), '_id': ObjectId('6683bc742368f131075b745d')}

The dictionary on the second line has an additional '_id' key. The application throws an exception when attempting to return this dictionary because FastAPI’s internal methods for converting Python dictionaries to JSON objects don’t know what to do with PyMongo’s ObjectIds.

There are different ways to fix this error, depending on the requirements of your application. The following solutions are presented in ascending order of complexity.

Solution 1: Remove the ID

If you don’t need to return the database event ID to users, remove the '_id' key from the dictionary before returning it. For example:

Click to Copy
@app.post("/events/")
async def create_event(event: Event):
    event_dict = event.dict()
    result = store_in_mongo("events", event_dict)
    event_dict.pop('_id', None) # remove key from dictionary if it exists

    if result:
        return event_dict
    else:
        return {"message": "Failed to create event."}

This solution hides the internal details of our data storage from users. Depending on the use cases and audience for your application, you may wish to hide event IDs from users or create your own, separate from the MongoDB document IDs.

Solution 2: Define a Custom JSONEncoder Class

We can define a custom JSONEncoder class that knows how to handle ObjectIds. For example:

Click to Copy
import json
from bson import ObjectId # bson = binary JSON, the data format used by MongoDB

class MyJSONEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, ObjectId):
            return str(o) # this will return the ID as a string
        return json.JSONEncoder.default(self, o)

To use this encoder, make the following change to create_event:

Click to Copy
@app.post("/events/")
async def create_event(event: Event):
    event_dict = event.dict()
    result = store_in_mongo("events", event_dict)

    if result:
        return MyJSONEncoder().encode(event_dict) # use custom JSONEncoder
    else:
        return {"message": "Failed to create event."}

This solution is the most complex, but also the most flexible, and could be useful if we need to use different JSONEncoders for different endpoints. This approach can be used to fix JSON encoding errors for any custom object.

  • Syntax.fmListen to the Syntax Podcast
  • Community SeriesIdentify, Trace, and Fix Endpoint Regression Issues
  • ResourcesBackend Error Monitoring 101
  • Syntax.fm logo
    Listen to the Syntax Podcast

    Tasty treats for web developers brought to you by Sentry. Get tips and tricks from Wes Bos and Scott Tolinski.

    SEE EPISODES

Considered “not bad” by 4 million developers and more than 100,000 organizations worldwide, Sentry provides code-level observability to many of the world’s best-known companies like Disney, Peloton, Cloudflare, Eventbrite, Slack, Supercell, and Rockstar Games. Each month we process billions of exceptions from the most popular products on the internet.

learn more about sentryjoin our discord
© 2024 • Sentry is a registered Trademark of Functional Software, Inc.