Upload a CSV file in FastAPI and convert it to JSON

David Y.

—March 15, 2024

The Problem

I’m building an endpoint for my FastAPI project that accepts a CSV file, converts the contents of that file to JSON, and returns that to the user. The returned JSON should be a list of dictionaries corresponding to the uploaded CSV file’s rows.

Whenever I try to upload a CSV file, I encounter the following error:

FileNotFoundError: [Errno 2] No such file or directory: 'example_data.csv'

Here’s my code:

from fastapi import FastAPI, File, UploadFile
import csv

app = FastAPI()

@app.post("/csv2json")
async def upload(file: UploadFile = File(...)):
    data = {}
    with open(file.filename, encoding='utf-8') as input_file:
        csvReader = csv.DictReader(input_file)
        for row in csvReader:
            data[row['ID']] = row

    return data

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

The CSV file looks like this:

ID,Name,Age,Occupation,Country
1,Alice,28,Engineer,USA
2,Bob,34,Doctor,Canada
3,Charlie,45,Artist,UK
4,Diana,23,Lawyer,Australia
5,Evan,36,Scientist,Germany

What’s causing this error and how do I fix it? My code is running on a Linux server.

The Solution

This error occurs because Python is attempting to open a file from a filename that does not exist on the disk. FastAPI’s UploadFile class, of which file is an instance, uses tempfile.SpooledTemporaryFile to represent uploaded files. This is a Python object that acts like a file in most ways but exists only in memory. Even though it has a filename attribute, it cannot be accessed using open, as it has not been written to disk. FastAPI does this to provide developers with flexibility in dealing with client-uploaded files and avoid the overhead of writing files to disk when not necessary.

On some systems, Python’s tempfile.TemporaryFile is an alias for tempfile.NamedTemporaryFile, which can be accessed using filenames. So this code may function correctly on some platforms, such as macOS and Windows.

To make our code truly cross-platform and avoid this error, we must access our uploaded file without relying on open. We can read the file’s contents as a bytes as follows:

file_bytes = file.file.read()

We can then decode the bytes in contents using the StringIO text stream from Python’s built-in io library:

buffer = StringIO(file_bytes.decode('utf-8'))

We can then use buffer in the same place the original code uses input_file:

from fastapi import FastAPI, File, UploadFile
import csv
from io import StringIO # new import

app = FastAPI()

@app.post("/csv2json")
async def upload(file: UploadFile = File(...)):
    data = {}

    # read file as bytes and decode bytes into text stream
    file_bytes = file.file.read()
    buffer = StringIO(file_bytes.decode('utf-8'))

    # process CSV
    csvReader = csv.DictReader(buffer)
    for row in csvReader:
        data[row['ID']] = row

    # close buffer and file
    buffer.close()
    file.file.close()

    # return JSON
    return data

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

This altered code should accept a CSV file and return a JSON version without errors.