01 / Applied AI & LLM evaluation
Single-agent vs. multi-agent LLM systems
Comparing agent architectures for business decision support through financial numerical reasoning.
Overview
For my master’s thesis, I explored whether adding specialized roles and verification steps to an LLM workflow can improve financial numerical reasoning, and what those gains cost in latency and API usage.
How it works
I designed four architectures, from a single-agent baseline to multi-agent workflows with specialized reasoning, verification, and coordination. I developed task-specific prompts, structured information flow, and evaluation procedures for FinQA questions.
The best configuration reached 34.70% execution accuracy compared with 32.17% for the single-agent baseline. I also evaluated program accuracy, reliability, latency and API cost, because accuracy alone is not a complete production decision.
What mattered
More agents do not automatically mean a better system. Designing evaluation around accuracy, complexity, and operational trade-offs matters as much as the orchestration itself.
Technologies
Public links will be added when available. The case study above describes the completed work.