On this page
How to Debug an AI-Generated API That Returns Wrong Data
Your AI built an API endpoint. It returns 200 OK with wrong data. Here's how to trace request → handler → response when you didn't write the handler — isolate the broken function, instrument it, and get a targeted fix.
Quick answer
- Isolate: which function in the handler is returning wrong data?
- Instrument: log the function’s inputs, outputs, and the data at each transformation step.
- Compare: expected vs actual at each step — the mismatch identifies the broken logic.
- Full system: How to Debug AI-Generated Code: A Complete System
The pattern: AI-generated endpoints that “work” but return wrong data
# AI-generated: returns 200, data is wrong
@app.get("/users/{user_id}/orders")
def get_user_orders(user_id: int):
user = db.get_user(user_id)
orders = db.get_orders()
filtered = [o for o in orders if o.user_id == user.id]
return {"orders": filtered}The endpoint returns 200. It returns JSON. But filtered is sometimes empty when it shouldn’t be. No error, no stack trace — just wrong data. This is the hardest kind of AI bug to debug because nothing crashes.
The trace: request → handler → response
For API endpoints, the debugging path is linear:
HTTP request
│
▼
Parse parameters/body
│
▼
Handler function ──► calls DB, external APIs, business logic
│
▼
Serialize response
│
▼
HTTP responseThe question: where does correct input become incorrect output?
Isolate: find the transformation that goes wrong
Add logging at each transformation boundary:
import logging
logger = logging.getLogger(__name__)
@app.get("/users/{user_id}/orders")
def get_user_orders(user_id: int):
logger.info(f"handler called: user_id={user_id}")
user = db.get_user(user_id)
logger.info(f"user from db: id={user.id}, name={user.name}")
orders = db.get_orders()
logger.info(f"orders from db: count={len(orders)}")
filtered = [o for o in orders if o.user_id == user.id]
logger.info(f"filtered orders: count={len(filtered)}, user_id={user.id}")
return {"orders": filtered}Now trigger the endpoint and read the logs:
handler called: user_id=42
user from db: id=42, name=Alice
orders from db: count=5
filtered orders: count=0, user_id=42 ← problem: filter removes all 5The bug is in the filtered line — the filter condition is removing orders when it shouldn’t. Now you ask the AI:
The filter `o.user_id == user.id` in `get_user_orders` returns 0
results when the database has 5 orders for user 42.
Expected: 5 orders
Actual: 0 orders
The orders from db have user_id=42. Show me the fix without
changing anything else.Common AI-generated API bugs and how to spot them
Silent type coercion
# AI generated: user_id comes from URL as string, compared to int
@app.get("/users/{user_id}/orders")
def get_orders(user_id): # type: str, not int!
orders = db.query("SELECT * FROM orders WHERE user_id = ?", user_id)
# Works in SQLite (automatic coercion), fails in Postgres (strict types)Spot it: log type(user_id) at the handler entry.
Query returns wrong shape
# AI assumed ORM returns dicts, but it returns tuples
orders = db.execute("SELECT id, total FROM orders").fetchall()
for order in orders:
print(order["id"]) # TypeError: tuple indices must be integersSpot it: log type(orders[0]) after the query.
Filter inverts the condition
# AI wrote the opposite condition
active = [u for u in users if not u.is_active] # should be: if u.is_activeSpot it: log len(users) before and len(active) after. If all users should be active but none pass the filter, the condition is inverted.
Data modified in place before return
orders = db.get_orders(user_id)
for o in orders:
o["total"] = calculate_discount(o["total"]) # modifies reference
# ... later code reassigns o["total"] back to originalSpot it: log orders right before return. The data was correct after processing but changed before serialization.
When the AI’s fix doesn’t work
If the AI produces a fix that doesn’t resolve the issue:
- Test in isolation: pull the broken function into a separate script, call it with the exact inputs from your logs, and verify the fix there before applying it to the codebase
- Ask for an explanation first: “Explain step by step what this function does with this input.” Understanding the logic often reveals the bug faster than getting the AI to guess at a fix
- Check for context the AI can’t see: the handler might depend on middleware, database schema, or configuration that the AI doesn’t know about — mention these explicitly
Quick reference
| Symptom | Likely cause | Check |
|---|---|---|
Returns [] when DB has data | Filter condition inverted or type mismatch | Log type(filter_field) and comparison operator |
| Returns wrong object shape | AI assumed wrong return type from ORM/DB | Log type(result) and first element |
| Returns stale data | AI cached result but didn’t invalidate | Look for @cache or lru_cache in handler |
Returns null/None | Exception swallowed by bare except | Remove try/except temporarily, read real error |
| Works on first call, fails on second | Mutable default arg or shared state | Check function signature for =[] or ={} |
Where this bites vibecoders
The AI’s endpoint returns 200. The vibecoder sees JSON, assumes it’s correct, and ships it. The bug is discovered when a user reports “my orders page is empty.” Tracing the handler with structured logging finds the broken transformation in minutes. Adding those logs takes 30 seconds.