Through countless conversations with customers and industry leaders, I’ve noticed a common theme: confusion about AI agents. I’ve been asked questions such as "What are agents?" "When should I use them?" or "How do they work in my environment?". These come up all the time. In this post, I’ll set the record straight and provide clarity on agents, drawing on my 10+ years of AI implementation experience. Whether you’re just starting or looking to enhance your use of agents, this guide is for you.
Agents are NOT new
First, let’s level set on one thing – Agents are NOT a brand-new concept. They have been around for years and have been implemented in industries across the globe. However, they are being perceived as a new or hot topic today because of the technological advancements that have been made. This is primarily driven by Large Language Models (LLMs) that have brought agents back into the spotlight. These advancements make agents more capable, flexible, and accessible than ever before.
What is an Agent?
Simply put, an agent is a system designed to perceive information and take action to achieve a specific goal. Examples include answering customer questions, executing transactions, or controlling smart devices in your home.

How AI Agents Have Evolved
Basic agents, such as virtual agents, have been around for years. In the early days, agents were simple and often rigid in their capabilities. However, modern AI agents powered by LLMs have taken a giant leap forward. They’ve helped significantly decrease the level of implementation effort to do similar tasks, work autonomously and more efficiently, and increase the art of the possible. Here are a few examples:
- Human-like interactions – Previously, chat conversations felt robotic and mechanical, which often frustrated users. Now, LLMs can provide fluid, conversational, and human-like experiences.
- Decision-making – Agents were reactive, performing pre-programmed actions or answering static FAQs. Modern agents can make realtime decisions based on learned preferences and adapt to changing conditions in real time.
Years ago, the level of effort to implement an agent required significant expertise, resources, and infrastructure. In modern day, low-code/no-code tools, pre-trained Large Language Models, and other modern cloud services make it much easier for enterprises to adopt and implement agentic solutions.
Then Vs Now: A Real Example
Back in 2016, when I was at IBM, I worked on a fascinating project for a global hair products company. The use case was to allow their employees access to sales and social media data. Their objectives were to better inform their product decisions, improve quality, and improve customer satisfaction.
One of the key questions the AI solution addressed was "what were my sales for Product X in the northeast region for Q2 of last year?". Our solution would use NLP to understand the question, populate one of many pre-defined SQL queries to retrieve the data from a SQL database, then include that information in a pre-defined dialog flow, along with a visual, back to the user. Let me break this down….we had:
- One agent to understand the end users’ questions via NLP
- One agent to execute SQL queries against a database to retrieve relevant data
- One agent to assemble the right dialog and data back to the chat
- One agent to create a visual (pie or line chart) to represent the data provided in the answer
- One agent to oversee and orchestrate all of these other agents
Yes, we achieved all this in production back in 2016! We had a total of 6 resources working on this, including myself as the engineering leader. Each resource focused on a specific "agent" capability.
If I was to deliver that same AI solution today, there would be a tremendous time savings on the implementation. I estimate that we would save at least 50% in build and delivery time. This is all thanks to the current technological advancements, with LLMs being the driving force. Had we done this today, we would:
- NOT need to hardcode and manage dialog flows
- NOT need to hardcode and manage SQL queries
- NOT need to spend exhaustive time training intents and entities as NLP is far more advanced today
Today’s LLM-powered agents accelerate in many areas such as natural language understanding which enables them to interpret unstructured data, make context-aware decisions, and carry out fluid, human-like conversations. This leap in capability allows modern agents to provide more accurate, personalized, and proactive support, moving beyond static functionality to dynamic, real-time problem-solving and decision-making.
How To Get Started
Before you try to make any technical or architectural decisions, there are several important factors you should consider first:
- Define Your Business Objective – what are you trying to achieve? (eg increase customer satisfaction or improve operational efficiency). This is critical and will be your constant reference point for everything you do. If this is not clear from the beginning, you run the risk of having a technical success but a business failure.
- Map Technology to the Use Case: Focus on the problem first. Let the technology serve the solution, not the other way around.
- Whiteboard Your Problem: Use approaches like Design Thinking to fully understand the use case, scenarios, and stakeholders.
- Ensure Data Readiness: Verify that your data is accessible, clean, and actionable (this is often a gap for many deployments).
Once you have these identified, you can begin to explore the technologies and architectures that best fit your needs. There are many other important aspects to consider and incorporate up front, such as data security and ensuring Responsible AI. It is critical that you embed these into your solution from the very beginning.
What’s Next
AI agents come in various types, such as simple, model-based, goal-based, utility-based, learning, and hierarchical agents. Their use cases extend far beyond conversational AI, supporting workflow automation, personalized recommendations, and much more. In future posts, I’ll dive into real-world examples and explain different agent types in more detail. For now, check out beginner-friendly resources from Microsoft and IBM to continue your learning journey. You can then check out Microsoft’s new AI Agent service and demo to explore how to get started.
Conclusion
AI agents are powerful tools that can transform how businesses operate and interact with customers. By understanding what they are and how they’ve evolved, you’ll be better prepared to harness their potential in your environment.
Ready to get started? Stay tuned for more insights, tips, and real-world examples!