Ready For AI Automation? Use A Large Language Model Agentic Workflow To Power Your Business Processes

The future of work is in automation and data-powered intelligence.

Seventy percent of companies are piloting automation technologies in at least one business unit.¹ Businesses today rely on artificial intelligence (AI) and large language models (LLMs) to automate complex tasks and enhance operational efficiency across functions. Powering that automation is agentic workflows, which use AI agents to execute the work.

Agentic workflows represent a paradigm shift from traditional automated processes, enabling a new level of autonomy and intelligence in business operations across all operational verticals. These workflows can be created to be self-sufficient, able to learn from interactions and make decisions without constant human oversight. We will look at the basics of agentic workflows, how they can revolutionize your operations, reduce costs and inaccuracies, and improve customer experience. By the end, you will have a clear idea of how you can incorporate LLM-driven agentic workflows into your business processes.

What Is an Agentic Workflow?

At its core, an agentic workflow is a process in which an LLM acts on behalf of users to perform tasks or provide assistance. These workflows leverage a model's capabilities to act as an intelligent intermediary between users and the information or services they require, enhancing productivity, efficiency and user experience. Agentic workflows have four types of design patterns:

• Reflection: LLM examines its own work for areas for improvement.
• Tool Use: LLM uses tools for certain tasks like gathering information, decision-making, processing information or taking action.
• Planning: LLM creates and executes a dynamic multi-step plan to reach a goal. This can be helpful with complex tasks that cannot be decomposed ahead of time.
• Multi-Agent Collaboration: Multiple agents work together on separate tasks to create a better solution.

A Typical Agentic Workflow – Customer Service Request

1. Input Reception: The LLM receives input from users or systems, such as text prompts, queries, commands, or requests for information or assistance.
   • Example: A customer sends a message to a customer service chatbot, stating, "I need to return a product I purchased."

2. Understanding and Analysis: The LLM analyzes the input to understand its intent, context, and any specific requirements or parameters associated with it.
   • Example: The LLM analyzes the message and determines the customer's intent is to initiate a return process.

3. Task Execution or Assistance: The LLM performs the necessary tasks to fulfill the user's request, which may include generating text, providing information or recommendations, performing calculations, or executing specific actions within integrated systems.
   • Example:
     • The LLM asks follow-up questions such as, "Could you please provide your order number?"
     • The customer responds with the order number.
     • The agent runs a tool that executes a function that retrieves the order details from the database (Tool Use design pattern).
     • Based on the return policy and order information, the LLM generates a return authorization code.
     • The LLM provides the customer with product return instructions and the return authorization code.

The agent might create a step-by-step plan to execute more complex tasks like helping a customer start a return for a defective product.

The agent might further break down the task into subtasks and delegate to agents with different roles: gathering information, initializing returns and customer support.

4. Feedback and Iteration: The LLM may provide feedback to the user or seek clarification, if necessary. It may also iterate on its responses based on user feedback or additional context provided during the interaction. This includes reflection, such as tool usage, planning, and delegation. The LLM in the agentic workflow helps select the tool(s) required to execute the task based on the textual context.
   • Example:
     • The customer may ask clarifying questions such as, "What is the return shipping address?"
     • The LLM determines that the agent must run the tool that executes a function that looks up and provides the return shipping address.
     • The customer confirms understanding.

After any step in the typical workflow, we can prompt the agent to critique its own work for correctness and efficiency, provide constructive feedback on how it can improve, and incorporate that feedback before or the next time it executes.

5. Completion and Follow-Up: Once the task is completed or the user's needs are addressed satisfactorily, the LLM may provide confirmation or follow-up information as needed. It may also update relevant records or systems to reflect the outcome of the interaction.
   • Example:
     • The customer initiates the return process following the provided instructions.
     • The LLM sends a confirmation message, "Your return request has been successfully submitted. You will receive further instructions via email."
     • The LLM logs the interaction and updates the customer's record with the return request details.

Levels of Autonomy in Workflows

As the example of a typical workflow illustrated, there are varying levels of interaction and autonomy within an agentic workflow. How you build it depends on your requirements and the resources available. Workflow autonomy is categorized into several levels, each offering increasing degrees of self-sufficiency and decision-making capabilities. You can think of these levels similarly to those of self-driving vehicles as defined by SAE International,² where level 0 is entirely manual, and level 5 represents full autonomy:

• Level 0 (Manual Operations): Workflows manually executed by human operators.
• Level 1 (Rule-Based Automation): Workflows follow predefined rules and conditions. Actions are triggered by specific inputs or events without real-time adaptation.
• Level 2 (Conditional Logic): Workflows incorporate basic decision-making capabilities based on conditional logic and predefined scenarios.
• Level 3 (Contextual Adaptation): Workflows use contextual data to adapt their actions. They can modify their behavior based on changes in their environment or task requirements.
• Level 4 (Machine Learning–Based Decisions): Workflows leverage models to make data-driven decisions. They can analyze patterns, predict outcomes, and optimize processes without explicit programming for each scenario.
• Level 5 (Fully Autonomous Operations): Workflows operate with minimal or no human intervention. They can understand complex tasks and, in time, can learn from their interactions to improve their efficiency.

The role of autonomy in agentic workflows is crucial for enhancing operational efficiency, reducing human intervention, and enabling the system to handle tasks that are dynamic, complex and require a high degree of intelligence.

How Agentic Workflows Benefit Operations

Agentic workflows offer numerous advantages over traditional automated systems. Without the need for human intervention, organizations can process larger amounts of work more accurately and quickly with agentic workflows. In practice, however, they are likely used to augment humans in the loop, freeing them up to complete more meaningful or impactful tasks. This autonomy leads to significant efficiency gains in process speed and resource capacity. Key benefits of agentic workflows include:

• Enhanced Efficiency: Significantly reduces the time and effort required to complete complex and repetitive tasks. This leads to faster processing times and higher throughput, enabling organizations to achieve more with fewer resources.
• Improved Accuracy: Leveraging AI and machine learning, can be developed to be less prone to human errors and can handle complex tasks that require precision and attention to detail.
• Versatility Across Domains: Can be applied to various industries and domains, from healthcare and finance to retail and manufacturing. Their flexibility and adaptability make them suitable for a wide range of applications, driving innovation and efficiency in different sectors.
• Enhanced User Experience: In customer-facing applications, can provide personalized and efficient service, leading to improved customer satisfaction and loyalty. They can handle customer queries, process requests and provide solutions in real time, enhancing the overall user experience.

Agentic workflows can also offer higher performance on tasks with smaller language models. For example, Andrew Ng, Joaquin Dominguez and John Santerre, in collaboration with Sequoia Capital, discovered that agentic workflows via GPT 3.5 can outperform non-agentic workflows with GPT 4, as depicted in Figure 1.³

Figure 1: Comparing Performance of Non-Agent vs. Agent LLM Applications

Additionally, Du et al. 2023 studied the impact of multiple agents in a workflow, which demonstrated that multiple agents outperform the use of single agents, as shown in Figure 2.⁴ NB: MMLU in the below chart is "Measuring Massive Multitask Language Understanding," a standard benchmark for evaluating the performance of language models, consisting of about 16,000 multiple-choice questions spanning over 50 academic subjects including mathematics, philosophy, law and medicine.

Figure 2: Performance of Single vs. Multi-Agent Workflows

The application of agentic workflows allows businesses to use smaller, cheaper and easier-to-maintain language models — lowering the cost and barrier to entry for generative AI projects.

How To Implement Agentic Workflows

Now that you know the fundamentals of agentic workflows leveraging LLMs, how do you implement them into your processes?

The easiest way to get started is to use an out-of-the-box package. There are many options in the market from which to choose. In our work and research, we have found that CrewAI, Microsoft's Autogen and LangChain's LangGraph are platforms that offer unique features and capabilities, making them suitable for different use cases and preferences. There are new providers and tools for agentic workflows constantly launching. The three we deep dive below represent a nonexhaustive list.

Limitations of Agentic Workflows

While agentic workflows with LLMs offer organizations significant benefits, take these considerations into account before implementing this technology in your organization:

• Steep Learning Curve: Implementing and managing LLM agentic workflows can be extremely complex and require specialized skills, adding to the learning curve.
• Inference Latency and Extra Compute Resources: Running LLMs and complex workflows can be computationally intensive, because of repeated calls to LLMs to complete workflow tasks. This can result in the need for significant processing power and storage capabilities, leading to higher operational costs and significantly higher latency for completing a query or set of tasks.
• Transparency Challenges: Understanding what agentic workflows are doing can be challenging, particularly with complex decision-making processes, raising concerns about accountability and bias.
• Dependency on Data Quality: The performance of agentic workflows is heavily dependent on the quality and quantity of data available, requiring significant effort for preprocessing and cleaning.
• Interoperability Issues: Ensuring interoperability between different components of agentic workflows and external systems can be challenging, particularly with proprietary technologies and diverse data formats.
• Scalability Constraints: As the complexity and number of workflows increase, so do the demands on computational resources, potentially leading to performance bottlenecks and increased costs.

Footnotes

1. McKinsey & Company. "Your questions about automation, answered." Survey report. July 8, 2022. https://www.mckinsey.com/capabilities/operations/our-insights/your-questions-about-automation-answered.

2. SAE International. "SAE Levels of Driving Automation Refined for Clarity and International Audience." SAE Blog, May 3, 2021. https://www.sae.org/blog/sae-j3016-update.

3. Sequoia Capital. "What's next for AI agentic workflows ft. Andrew Ng of AI Fund." YouTube video, 13:18. June 22, 2023. https://www.youtube.com/watch?v=sal78ACtGTc.

4. Qian, Yujia, Xuhui Zhou, Yixuan Su, Liang Zhou, Shuang Xu, Huan Sun, and Yiming Yang. "ChatDev: Conversational AI for Software Development." arXiv preprint arXiv:2305.14325 (2023). https://arxiv.org/pdf/2305.14325.

Originally published 15 July 2024

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.