This follows on from Python typings packages breaks FastAPI on Vercel - #3 by brooke-tintunacom.
After I fixed that problem (using python package for Typing) and solving some other problems I now have a problem where the State I’m storing does not exist when the app runs on Vercel, however it works fine when running locally. Any ideas?
The code is:
$ cat requirements.txt
python-dotenv
openai
instructor
rich
pydantic
fastapi
mangum
uvicorn
atomic-agents
$ cat vercel.json
{
"version": 2,
"builds": [
{
"src": "app/main.py",
"use": "@vercel/python"
}
],
"routes": [
{
"src": "/(.*)",
"dest": "app/main.py"
}
]
}
$ cat app/lifespan.py
from contextlib import asynccontextmanager
from app.llm.atomic_agents_service import AtomicAgentsService
@asynccontextmanager
async def lifespan(app):
# Load the LLM service - do it this way so that it is loaded once and not on every request
atomic_agents_service = AtomicAgentsService()
print(" - created atomic_agents_service: ", atomic_agents_service)
app.state.atomic_agents_service = atomic_agents_service
yield
# Clean up the LLM service and release the resources
app.state.atomic_agents_service = None
$ cat app/main.py
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from mangum import Mangum
from .routes import router as api_router
from app.lifespan import lifespan
app = FastAPI(lifespan=lifespan)
# CORS configuration
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # Allows all origins
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(api_router)
@app.get("/")
async def read_root():
return {"health": "up"}
$ cat app/routes.py
from app.atomic_agents_data import AtomicAgentMemoryModel
from app.llm.atomic_agents_service import AtomicAgentsService
from rich.console import Console
import json
router = APIRouter()
console = Console()
@router.post("/choice")
async def post_choice(request: Request, choice: str = None, memory: AtomicAgentMemoryModel = None):
console.print("/choice")
console.print(" - choice: ", choice)
console.print(" - memory: ", memory.json()) if memory else console.print(" - memory: None")
console.print("IN BETWEEN")
console.print(" - request.app.state: ", request.app.state)
console.print(" - request.app.state dir: ", dir(request.app.state))
console.print(" - request.app.state state: ", request.app.state._state)
console.print(" - atomic_agents_service: ", request.app.state.atomic_agents_service)
# `memory` will automatically be deserialized into an AtomicAgentMemoryModel instance
if (memory is None or memory.history is None or len(memory.history) == 0):
response = request.app.state.atomic_agents_service.start()
else:
response = request.app.state.atomic_agents_service.post_choice(choice, memory)
# Perform your logic here
return {"request": choice, "response": response}
Locally the terminal shows:
AtomicAgentsService - init
- created atomic_agents_service: <app.llm.atomic_agents_service.AtomicAgentsService object at 0x1087fd940>
...
/choice
- choice: None
- memory: None
IN BETWEEN
- request.app.state: <starlette.datastructures.State object at 0x1087cf0e0>
- request.app.state state:
{'atomic_agents_service': <app.llm.atomic_agents_service.AtomicAgentsService object at 0x1087fd940>}
- atomic_agents_service: <app.llm.atomic_agents_service.AtomicAgentsService object at 0x1087fd940>
But the vercel runtime log shows (just request and message):
# NOTE --> " - created atomic_agents_service: ..." is misssing <-- NOTE
/choice 127.0.0.1 - - [30/Nov/2024 04:44:52] "POST /choice HTTP/1.1" 500 -
/choice. - request.app.state state: {}
/choice. - request.app.state: <starlette.datastructures.State object at 0x7f5d466da810>
/choice. IN BETWEEN
/choice - memory: None
/choice. - choice: None
It looks like the lifespan is not executing. There is no - atomic_agents_service: xxxxx in the Vercel runtime log.
Any ideas?
