Building a Conversational Agent for job aids (RAG Use Case)
What if you could turn dense documentation into an always-on assistant that provides quick, contextual answers to complex questions? An agent that lives and breathes the domain and shares always up to date information with its users. That was the idea behind creating an AI-powered conversational agent that we created at a large Utility customer— and with the Microsoft Co-Pilot studio, this was a task made super easy.
Step 1: Focus on the Need
Every project begins with a problem to solve. For this conversational agent, the challenge was clear: team members needed reliable, on-demand answers to questions without the friction of searching through countless documents.
The Question:
How can we reduce the time spent looking for information and increase the confidence in the answers provided?
The solution had to:
- Understand the nuances of specific processes.
- Deliver clear, accurate answers based on trusted documentation.
- Be intuitive and fast enough for everyday use.
The first step was to describe the agent’s purpose. Microsoft Co-Pilot Studio provided an intuitive interface for this, as seen in the "Describe Your Agent" screenshot. Using everyday language, I clarified the agent’s role to help users navigate their job aids efficiently.
“I’d like to create an Agent that can read up all the job aids from specific folders and answer business questions around how the process works, the use of applications and point to relevant job aids where necessary”
Step 2: Build with Simplicity
Microsoft Co-Pilot Studio made the process refreshingly straightforward. The interface guided me through setting up the agent in just a few steps:
- Name and Purpose: I named the agent "JobAidAgent" and defined its role, as seen in the "Agent Details" screenshot.
- Knowledge Integration: I uploaded the documents and configured the agent to use them as the source of truth.
- Response Confidence: I instructed the agent to always provide a confidence score, ensuring users could trust its responses.
The user-friendly platform made it easy to create a robust conversational interface without extensive technical expertise.
“Enter the following data or whatever meets your needs.”
Step 3: Add the Knowledge Base
Instead of diving into complex integrations or custom code, With Co-Pilot I could simply upload a collection of job aids and process documents. These became the foundation of the agent’s knowledge.
The Approach:
Microsoft Co-Pilot Studio allowed me to upload documents directly to build the agent’s knowledge base. The "Add Knowledge" screenshot shows how I could seamlessly include files
By enabling AI to use these documents, the agent was equipped to retrieve and synthesize information effectively.
“Upload all the files you need to simply link to an existing SharePoint link”
Step 4: Test, Iterate, Refine
The real magic happened during testing. Co-Pilot Studio’s built-in testing environment allowed me to simulate questions like, "How do I update a revenue class?" The agent’s responses were not only accurate but structured step-by-step, as shown in the "Test Your Agent" screenshot:
- Log in to the system with your credentials.
- Navigate to the "Revenue Class Management" section.
- Select and edit the relevant revenue class.
- Make necessary changes and save updates.
This iterative process revealed areas for refinement, such as improving the agent’s handling of ambiguous queries or prompting for more context when needed.
“And begin to test your Agent. Try many variations to fully test how the model is working and then publish the model”
Lessons Learned
This project reinforced an important lesson: simplicity scales. By starting with a clear focus and leveraging tools like Microsoft Co-Pilot Studio, I avoided unnecessary complexity while delivering immediate value.
Challenges Overcome:
- Overlapping Contexts: Clear segmentation of topics ensured the agent retrieved the most relevant information.
- Ambiguity in Queries: Training the agent to ask clarifying questions improved user satisfaction.
- Knowledge Gaps: Escalation workflows ensured unresolved questions didn’t become roadblocks.
.png)
When “Transformation” Isn’t Transformation
There’s a quiet absurdity in how most organizations approach change. Every few years, a vendor announces the end of support for a version — and suddenly, an entire enterprise scrambles into “transformation mode.” Slide decks get rewritten, budgets inflate, timelines stretch, and the word strategy gets thrown around like confetti.
Utilities Billing — Don't let Watermelon Reporting fool you. Billing sees the red everyday.
When the dashboards look green but your billing team is buried in exceptions—there’s a disconnect.In utilities, billing becomes the silent sinkhole of revenue operations, absorbing every upstream delay, data miss, and failed process. We call it Watermelon Reporting—green on the outside, red on the inside.This post is an ode to the teams who keep the lights on (and the bills going out) despite the chaos—and how the Celonis Utilities Smart Billing App helps them shift from firefighting to forward-looking.
My Journey to Building the App I Knew Utilities Needed — Part 3: Why the Squeaky Wheel Keeps Winning
In Utilities, being loud pays.If you're the one complaining the most, chances are, your issue jumps the queue. But what happens to the silent failures no one’s yelling about?That’s the question I dive into in Part 3 of my series.We’ve all seen it—leadership pulled into fire after fire, solving the same problems over and over, while deeper issues go unnoticed. Escalation becomes the strategy. And soon, the whole organization is wired for noise instead of insight.In this post, I talk about how we deliberately built our app to flip that model.
My journey to building an App I knew Utilities needed — Part 2 (The Beginning)
In Part 2 of My Journey to Building an App I Knew Utilities Needed, I explore how process mining helped a utility uncover the real causes of delays in the New Connection process—insights that traditional BI tools simply couldn't deliver. This post highlights how utilities can move beyond surface-level reporting to truly understand operational bottlenecks, improve customer experience, and strengthen their rate cases with data-driven transparency.
My journey to building an App I knew Utilities needed — Part 1
"For years, utilities have struggled with process inefficiencies that lead to millions in fines, regulatory scrutiny, and a complete breakdown in customer trust. But what if there was a way to cut through the chaos and truly understand what's going wrong? In this first part of my journey, I share how process mining and the Celonis platform opened my eyes to the brutal truth: most companies don't even recognize their real processes. I’m excited to share how my company, The Good Quarter, is on the verge of launching an app that will finally give utilities the insight they need to fix their meter-to-cash process—and stop the cycle of missed opportunities and massive costs.
A Blockbuster in AI Assistance: Google AI Studio's Screen-Sharing Feature
Discover the groundbreaking new feature in Google AI Studio that lets you share your screen with AI and get real-time, voice-driven assistance. Say goodbye to screenshots and miscommunication—this tool transforms productivity, troubleshooting, and learning by offering seamless, contextual guidance. It's like having J.A.R.V.I.S. from Iron Man as your personal assistant!
Everything, Everywhere, All at Once: The World of Object-Centric Process Mining
Discover how Object-Centric Process Mining (OCPM) with Celonis transforms business operations. Uncover multi-object interactions, hidden relationships, and end-to-end transparency to drive actionable insights and operational efficiency. Learn how Celonis empowers businesses to move beyond traditional process analysis
Start with the Question: Turning Data into Insight
Before you touch the data, ask yourself: What do I really want to know? Forget about what the data is; think about what you’re trying to uncover. A good question is specific and meaningful, not just to you but to anyone looking for answers.
Embracing the Opportunities of SAP S/4 HANA Transformations
You hear a lot about the technology risks around S/4 Implementation projects.
What no one is really talking about are the business challenges and what the business can do to address 3 of the most common factors that lead to chaotic and unsuccessful S/4 HANA implementations
Rethinking Metrics: The Power of Object-Centric Process Mining
Discover the transformative power of Object-Centric Process Modeling (OCPM) in business operations. Unlike traditional tools, OCPM provides a dynamic, interconnected view of processes, revealing insights that drive efficiency and strategic alignment. Learn how OCPM is reshaping process analytics and empowering businesses to achieve operational excellence.
Who owns your data?
Do you own your data? What aspects of your data do you control? What areas are you responsible for in your organization? This article explores some of these concepts.
Getting better at what you do by how you do it
There are myriad ways of delivering greater efficiencies to the business. Some rely on incremental changes to deliver value in small increments. In contrast, others require a greater degree of change for an organization.
Sometimes, a large transformation project is an excellent opportunity to restructure your business operations. And one of the ways to consider what that aspirational state should be is by leveraging the principles of Domain Driven Design.
Move aside traditional methods.
It’s 2024. I think the continuous improvement folks could use a taste of their own medicine. I’ve outlined a few ways process mining could be used to improve the traditional process mapping methods
Why you track your life, but not your processes?
I’m confident you can pick up your smart phone and piece together a significant portion of your life in the past day, month, even years! Can you say the same about your processes at work?
Data Insights to Actionable Signals to Desired Outcomes
Organizations rely on data insights in today's data-driven world to inform their decision-making processes and drive successful outcomes. However, simply collecting and analyzing data is not enough.
We explore the journey from data to insight to action to outcome and what organizations must focus on along the way.
Top 3 reasons why data, analytics and insight are all dressed up and have nowhere to go in your enterprise
In neurology, a condition called ataxia manifests as a lack of muscle control or coordination of voluntary movements, which may be an apt metaphor for this dissonance in the business organization in relation to data.
So what is the solution? What can business leaders do to turn the tide on this trend?