Skip to main content

Adding usage telemetry to the LLM adapter broke every call site silently

What happened: to track token spend, call_chat and call_embed were changed to return two values instead of one โ€” the result and a usage dict. The ~8 call sites in server.py weren't updated, and Python lets both values land in a single variable without erroring:

response = call_chat(messages)   # was: "the model's text"
# now: ("the model's text", {"tokens": 120})
parse_clarify_result(response) # expects a string, gets a tuple -> breaks here

So nothing failed at the call itself. The wrong value was passed on and broke later, in the JSON parser and in pgvector search โ€” nowhere near the line that changed.

Fix: unpack and discard the usage dict at every call site, in one pass.

vectors, _ = call_embed([story_text])
response, _ = call_chat(messages, model=config.CHAT_MODEL)

Why it matters: every caller depends on the exact shape the adapter returns, so adding telemetry to it breaks all of them โ€” it isn't an internal detail. Returning a tuple is the version that breaks silently; returning one object fails loudly instead.

Reference: The provider adapter layer โ€” the adapter is the right place to capture usage, and this is the cost of changing its shape after callers depend on it.