Ask a coding assistant how to add two numbers in JavaScript and you get something like:

function add(a, b) { return a + b; } 

In BeeAI Framework, an answer like that used to stop the agent run. The model was right. The hard part was reading its answer.

How the final answer is modelled

BeeAI's RequirementAgent and ToolCallingAgent treat the final answer as one more tool: final_answer, with a pydantic schema (FinalAnswerToolSchema, or your own expected_output schema). Every step is a tool call, and every tool call is validated. That keeps the loop simple.

Models don't always play along, though. Sometimes they reply in plain text. So the agent converts plain text into a final_answer tool call.

The old conversion, roughly:

match = first_curly_block(text) # first {...} in the reply if match: tool_input = parse_broken_json(match) # lenient repair parser else: tool_input = {"response": text} 

parse_broken_json is lenient by design. It repairs whatever it gets and returns a value. But a value isn't necessarily an object, and nothing checked the result against the schema before it became the tool input.

Three correct answers, three failures

Answer What the parser returned What happened
function add(a, b) { return a + b; } a list Tool input must be an object, so the call couldn't be built and the run stopped
Use the placeholder {name} in your template. a list Same
Set the config to {"port": 8080} and restart. {"port": 8080} Valid JSON, invalid final answer. The model was told it made an error; the original text was gone

For the third case, with a scripted model that repeats its answer, the agent kept retrying until max_iterations: 96 model calls for one question (156 in the other final_answer_as_tool mode). A real model may recover, but by then the right answer was already discarded and you'd paid for the extra calls.

The second problem: the error path

When a tool fails, Tool.run emits a ToolErrorEvent(input=...) inside its except block. ToolErrorEvent.input was typed BaseModel | dict.

So when the tool input was a list, building the error event raised a pydantic ValidationError. That isn't a ToolError, so run_tool didn't catch it, and the whole run aborted. This hit any tool called with JSON-array arguments, not only final_answer.

The code meant to report the problem became the problem, and the model never got feedback it could act on.

The fix

One rule: only treat text as structure if it passes the schema.

def text_to_final_answer_input(text, schema): parsed = try_parse_first_object(text) if schema is FinalAnswerToolSchema: if parsed is not None and is_valid(parsed, schema): return parsed return {"response": text} # keep the answer exactly as written # custom expected_output: pass JSON objects on (useful validation errors), # but never lists or plain values return parsed if isinstance(parsed, dict) else {"response": text} 

(Simplified; the real helper lives in the PR.)

  • Both agents now use text_to_final_answer_input.
  • ToolErrorEvent.input accepts any value, so a badly shaped tool call becomes a tool error the model can see and fix, and the run continues.

No public API changed. 23 tests cover the helper, both agents, both final_answer_as_tool modes, and array-argument tool calls; 16 of them fail on the old code. The PR merged on November 12, my first merged Python contribution.

Takeaways

  1. "Is this structured?" and "is this valid?" must use the same schema. If the first check is looser, it quietly passes the wrong thing to the second.
  2. Lenient parsers need a strict gate after them. Repairing JSON is fine; trusting the repaired value without validation isn't.
  3. Test the error path with strange input. A list, a string, null. An error handler that can throw turns a recoverable mistake into a stopped run.

The model is often fine. What breaks is the code that reads its output.

What's the strangest correct answer you've seen your agent reject?