On this page
  1. The pattern: AI-generated endpoints that “work” but return wrong data
  2. The trace: request → handler → response
  3. Isolate: find the transformation that goes wrong
  4. Common AI-generated API bugs and how to spot them
    1. Silent type coercion
    2. Query returns wrong shape
    3. Filter inverts the condition
    4. Data modified in place before return
  5. When the AI’s fix doesn’t work
  6. Quick reference
  7. Related topics
how-to

How to Debug an AI-Generated API That Returns Wrong Data

Your AI built an API endpoint. It returns 200 OK with wrong data. Here's how to trace request → handler → response when you didn't write the handler — isolate the broken function, instrument it, and get a targeted fix.

Quick answer

  • Isolate: which function in the handler is returning wrong data?
  • Instrument: log the function’s inputs, outputs, and the data at each transformation step.
  • Compare: expected vs actual at each step — the mismatch identifies the broken logic.
  • Full system: How to Debug AI-Generated Code: A Complete System

The pattern: AI-generated endpoints that “work” but return wrong data

# AI-generated: returns 200, data is wrong
@app.get("/users/{user_id}/orders")
def get_user_orders(user_id: int):
    user = db.get_user(user_id)
    orders = db.get_orders()
    filtered = [o for o in orders if o.user_id == user.id]
    return {"orders": filtered}

The endpoint returns 200. It returns JSON. But filtered is sometimes empty when it shouldn’t be. No error, no stack trace — just wrong data. This is the hardest kind of AI bug to debug because nothing crashes.

The trace: request → handler → response

For API endpoints, the debugging path is linear:

HTTP request


Parse parameters/body


Handler function ──► calls DB, external APIs, business logic


Serialize response


HTTP response

The question: where does correct input become incorrect output?

Isolate: find the transformation that goes wrong

Add logging at each transformation boundary:

import logging
logger = logging.getLogger(__name__)

@app.get("/users/{user_id}/orders")
def get_user_orders(user_id: int):
    logger.info(f"handler called: user_id={user_id}")

    user = db.get_user(user_id)
    logger.info(f"user from db: id={user.id}, name={user.name}")

    orders = db.get_orders()
    logger.info(f"orders from db: count={len(orders)}")

    filtered = [o for o in orders if o.user_id == user.id]
    logger.info(f"filtered orders: count={len(filtered)}, user_id={user.id}")

    return {"orders": filtered}

Now trigger the endpoint and read the logs:

handler called: user_id=42
user from db: id=42, name=Alice
orders from db: count=5
filtered orders: count=0, user_id=42        ← problem: filter removes all 5

The bug is in the filtered line — the filter condition is removing orders when it shouldn’t. Now you ask the AI:

The filter `o.user_id == user.id` in `get_user_orders` returns 0
results when the database has 5 orders for user 42.

Expected: 5 orders
Actual: 0 orders

The orders from db have user_id=42. Show me the fix without
changing anything else.

Common AI-generated API bugs and how to spot them

Silent type coercion

# AI generated: user_id comes from URL as string, compared to int
@app.get("/users/{user_id}/orders")
def get_orders(user_id):                  # type: str, not int!
    orders = db.query("SELECT * FROM orders WHERE user_id = ?", user_id)
    # Works in SQLite (automatic coercion), fails in Postgres (strict types)

Spot it: log type(user_id) at the handler entry.

Query returns wrong shape

# AI assumed ORM returns dicts, but it returns tuples
orders = db.execute("SELECT id, total FROM orders").fetchall()
for order in orders:
    print(order["id"])       # TypeError: tuple indices must be integers

Spot it: log type(orders[0]) after the query.

Filter inverts the condition

# AI wrote the opposite condition
active = [u for u in users if not u.is_active]    # should be: if u.is_active

Spot it: log len(users) before and len(active) after. If all users should be active but none pass the filter, the condition is inverted.

Data modified in place before return

orders = db.get_orders(user_id)
for o in orders:
    o["total"] = calculate_discount(o["total"])   # modifies reference
# ... later code reassigns o["total"] back to original

Spot it: log orders right before return. The data was correct after processing but changed before serialization.

When the AI’s fix doesn’t work

If the AI produces a fix that doesn’t resolve the issue:

  1. Test in isolation: pull the broken function into a separate script, call it with the exact inputs from your logs, and verify the fix there before applying it to the codebase
  2. Ask for an explanation first: “Explain step by step what this function does with this input.” Understanding the logic often reveals the bug faster than getting the AI to guess at a fix
  3. Check for context the AI can’t see: the handler might depend on middleware, database schema, or configuration that the AI doesn’t know about — mention these explicitly

Quick reference

SymptomLikely causeCheck
Returns [] when DB has dataFilter condition inverted or type mismatchLog type(filter_field) and comparison operator
Returns wrong object shapeAI assumed wrong return type from ORM/DBLog type(result) and first element
Returns stale dataAI cached result but didn’t invalidateLook for @cache or lru_cache in handler
Returns null/NoneException swallowed by bare exceptRemove try/except temporarily, read real error
Works on first call, fails on secondMutable default arg or shared stateCheck function signature for =[] or ={}

Where this bites vibecoders

The AI’s endpoint returns 200. The vibecoder sees JSON, assumes it’s correct, and ships it. The bug is discovered when a user reports “my orders page is empty.” Tracing the handler with structured logging finds the broken transformation in minutes. Adding those logs takes 30 seconds.


Share: