Multi-Agent AI Systems

Our recent topic on Retrieval Augmented Generation (RAG) sparked many questions, including a request to highlight what multi-agent systems are.

This is an exciting topic because agents, in general, and multi-agent systems are considered on the frontline of current development in GenAI.

Multi-Agent Systems in FinTech and Banking

Last time, I heard many, “Wow, interesting... but I’m lost!”

So, let’s start with some real-world examples to make it clear. Imagine AI agents as your digital team, handling loan approvals, fraud checks, and customer support - all at once. They collaborate, adapt, and even make decisions so your mobile banking app runs smoother than ever.

For instance, consider an AI assistant that qualifies loan applicants based on real-time data from multiple sources. Or a team of agents coordinating customer support, fraud detection, and risk management, reducing downtime and operational costs. Multi-agent structures help these “digital bankers” interact dynamically, scaling services like mobile banking, fraud detection, and customer support in real-time.

Several more examples:

1. Fraud detection. Imagine agents constantly monitoring transactions. When they spot something suspicious, they alert another agent to flag the account, while a third verifies identity - all in seconds, minimizing fraud risks.

2. Digital advisors act as personal finance coaches. One agent analyzes spending, another tracks investments, and a third offers personalized tips based on user goals.

3. Agents can handle customer support queries around the clock. They triage issues, escalate when needed, and even collaborate to solve complex problems.

4. Multi-agent systems can ensure KYC and compliance by verifying user identity, cross-referencing data with external sources, and flagging issues to a compliance officer agent in the background.

5. Agents assess market data, adjust credit limits, and recommend portfolio adjustments based on real-time financial trends, helping banks stay ahead of risks.

These agents work together, autonomously and intelligently, to make digital banking faster, safer, and more responsive. Multi-agent systems facilitate seamless and scalable solutions, allowing each agent to specialize, collaborate, and adapt instantly based on the context.

What are Agents?

Agents are LLM-powered workers who can solve complex, multi-step problems that require robust decision-making and interaction with various external data sources. Agents concept is very close to RAG. An agent might use RAG tooling to fulfil tasks, but there’s a main difference between them: agents implement and use reasoning loops.

A reasoning loop is a brand-new concept which was introduced relatively recently. In a reasoning loop, the agent can break down a complex task into sub-tasks and solve them step by step while maintaining a conversational memory to keep the context of execution and, finally, craft a final answer to the user request.

There are two primary expressions of reasoning:

• Reasoning through evaluation is an ability to effectively breakdown a problem by iteratively planning, assessing progress, and adjusting its approach until the task is completed

• Reasoning through tools is the ability to decide which tools to call and what parameters to call it. This is, basically, a RAG scenario, but instead of a fixed tool sequence, an agent builds that dynamically, using some preliminary training and tool definitions.

Almost all modern LLM models could do both types of reasoning (though tool usage usually needs extra training and come with separate instruct branches of the main model), but, of course, their skill in that is very much dependent on model architecture.

For example, OpenAI’s new o1 model is the best at reasoning through evaluation and planning because it heavily relies on Chain of thought pattern usage. This has significantly improved its ability to think through and solve complex challenges. OpenAI claims that the o1 model has surpassed human PhD-level accuracy on the GPQA and scored in the 86th-93rd percentile on Codeforces contests[1], though there’s no independent confirmation on that. But, at the same time, now this model is not instructed on how to use any tools, so it could not be used in RAG scenarios.

However, size does matter here. For example, according to the Llama 3.1 model card, the Llama 3.1-8B model is unreliable for tasks that involve maintaining a conversation and calling tools. Instead, larger models with 70B+ parameters should be used for these tasks.

Both types of reasoning are powerful independently, and when combined, they have the potential to create agents that can autonomously interact with other agents and the environment.

Multi-agent systems

Multi-agent systems are a novel approach that aims to enhance single LLM agents' capabilities by leveraging collaborations among agents and their specialized abilities[2].  Each agent has distinctive capabilities and roles in multi-agent systems, collaborating to fulfil some common objectives. This has proven highly effective for tasks requiring deep thought.

Multi-agent systems involve agents of various specializations and orchestrate complex interactions among them. Shared memory is required to enable agents to access some common context, retrieve information necessary for their tasks, store experience and adapt their strategies.

There are multiple ways to build multi-agent systems. They are usually categorized  into various types based on each agent’s functionality and their interactions[3]:

• Equi-Level Structure. LLM agents in an equi-level system operate at the same hierarchical level, where each agent has its own role and strategy. The agents in such systems can have the same, neutral, or opposing objectives. The emphasis is on collective decision-making and shared responsibilities.

• Hierarchical Structure. Hierarchical structures typically consist of a leader and one or multiple followers. The leader’s role is to guide or plan while the followers respond or execute based on the leader’s instructions. Hierarchical structures are prevalent in scenarios where coordinated efforts directed by a central authority are essential.

• Nested Structure. Nested or hybrid structures constitute sub-structures of equi-level and/or hierarchical structures in the same multi-agent system. The “big picture” of the system can be either equi-level or hierarchical. As some agents have to handle complex tasks, they break them down into small ones, construct a sub-system, either equi-level or hierarchical, and “invite” several agents to help with those tasks.

However, there are multiple significant challenges, and, to date, they still need to be addressed. They are:

• Skill mapping: lack of a standardized way to publish and explore an agent’s unique skills

• Ambiguity in Language: each LLM might generate responses that can be interpreted in multiple ways, leading to misunderstandings between agents

• Contextual Misalignment: Agents may operate under different contextual assumptions, which can result in conflicting actions or decisions

• Standardizing Communication Protocols: Establishing clear protocols for agent communication can mitigate misunderstandings and enhance collaboration

• Performance evaluation: Defining and applying fairness metrics can help evaluate the performance of LLMs in multi-agent systems

• Optimal task allocation: Optimizing task allocation to leverage agents’ unique skills and specializations

• Robust reasoning: Fostering robust reasoning through iterative debates or discussions among a subset of agents to enhance intermediate results

• Single context management: Managing complex and layered context information, such as context for overall tasks, single agents, and some common knowledge between agents, while ensuring alignment to the general objective

• Memory management: Managing various types of memory that serve different objectives coherently to the interactions in multiagent systems.

Conclusion

Multi-agent systems are a truly promising technology that will eventually lead to fully autonomous decision-making and problem-solving across diverse domains. These systems can collaborate, coordinate, and adapt dynamically to tackle intricate tasks that would be challenging for individual agents.

However, significant challenges remain, which researchers and developers have yet to address. With continued research and innovation, multi-agent systems have the potential to unlock new possibilities, ultimately leading to more autonomous and intelligent systems capable of enhancing human decision-making and improving outcomes across various industries.

Are you curious about how multi-agent systems could transform your fintech or digital banking operations? Drop a comment with your thoughts on possible application scenarios! Or DM me to explore how we can tailor these solutions to your needs.

[1]     ‘In awe’: scientists impressed by latest ChatGPT model o1, 10/01/2024, Nature.com

[2]     Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents by Yashar Talebirad, Amirhossein Nadiri, Jun 2023, arXiv.org

[3]     LLM Multi-Agent Systems: Challenges and Open Problems, Shanshan Han, Qifan Zhang et all, Feb 2024, arXiv.org