From RAG to Agentic Workflows: Building an AI-Native CRM That Can Act

Introduction

A few weeks ago, I started with a simple question:

What would it take to make a CRM feel less like a database and more like an assistant?

The first version of the experiment focused on Gmail.

Emails were ingested, stored, converted into knowledge items, chunked, embedded, and indexed in PostgreSQL with pgvector. A retrieval pipeline could then find relevant information and provide it to an LLM to answer questions.

That was my first important milestone: the system could answer questions from my data rather than asking me to search for the data myself.

But another question appeared immediately.

What happens when answering a question requires more than retrieval?

Suppose I ask:

"Tell me everything important about my relationship with Salesforce, check the current opportunity, and tell me what I should follow up on."

There may be relevant information in emails, knowledge items, opportunities, notes, and eventually many other systems.

The user should not have to know where that information lives.

The system should.

That became the next step in the experiment.

RAG retrieves. An agent can decide what to retrieve, what else it needs, what action to take, and when it should involve the user.

This article is about that transition.

The Starting Point: RAG Was Only the Beginning

The previous iteration of the system established the knowledge pipeline:

The backend was deliberately separated into services responsible for Gmail integration, email storage, knowledge creation, chunking, embeddings, and search.

Events connected the ingestion stages rather than making every component directly dependent on every other component.

The result was a useful RAG foundation.

The system could answer a question such as:

"Do I have any emails discussing the wine industry?"

without requiring me to manually search Gmail.

That is already a significant change in interaction.

But it still has a fundamental limitation.

The user is responsible for knowing what to ask the retrieval system.

The Cognitive Problem

A database exists partly because humans cannot reliably remember everything.

But a conventional database still requires humans to:

• know what information exists,

• know where it exists,

• enter information,

• keep it updated,

• search for it,

• interpret it,

• connect it with other information,

• and decide what to do next.

That is a lot of cognitive work.

This became particularly relevant to the CRM I was building.

The product thinking had already moved away from a traditional lead-centric CRM toward an opportunity-centric system.

Conversations with CRM users reinforced another observation: people do not particularly enjoy maintaining CRM data. They want to spend their time building relationships, closing business, communicating with customers, and making decisions.

The CRM should therefore reduce cognitive load rather than create another administrative workload.

That led to a different product question:

What if the system could do more of the thinking between the user's intention and the system's data?

That is where the agent enters.

From Query Processing to Intent Processing

A traditional RAG application looks roughly like this:

An agentic system changes the relationship:

The important change is not that the LLM became "smarter".

The application gave the LLM capabilities.

Instead of exposing the database directly, I exposed business capabilities through tools.

For example:

The agent does not need to know that an opportunity is stored in a PostgreSQL table.

It needs to know:

"If I need the current opportunity, I have a capability that can retrieve it."

That is a much cleaner abstraction.

Why I Introduced a Separate Python Agent Service

The original backend is written in Go.

That remains the right place for the core application responsibilities:

• authentication,

• authorization,

• integrations,

• API handling,

• data storage,

• CRUD operations,

• ingestion,

• embeddings,

• retrieval,

• and business services.

The agent layer introduced a different set of concerns:

• prompts,

• tool definitions,

• conversation state,

• agent execution,

• tool selection,

• human-in-the-loop behavior,

• and eventually more sophisticated orchestration.

I therefore introduced a separate Python service.

This is not microservices for the sake of saying "microservices".

The purpose of the boundary is independent evolution.

The agent service can evolve its orchestration model without forcing the core backend to understand the details of LangChain, LangGraph, prompts, or agent state.

At the same time, the backend remains the system of record.

This separation also leaves room for the agent layer to evolve independently—from a simple tool-using agent to explicit LangGraph workflows and eventually more specialized agents.

Why Python?

The decision was not that Go is unsuitable for AI.

Go is perfectly capable of calling models, generating embeddings, interacting with vector databases, and implementing agentic workflows.

In fact, the original RAG implementation demonstrated that.

The reason for introducing Python was different.

The AI ecosystem moves quickly, and Python currently provides a particularly rich environment for experimentation with:

• LLM integrations,

• agent frameworks,

• document processing,

• tool orchestration,

• structured outputs,

• evaluation,

• and AI-specific libraries.

I wanted the core application and the AI orchestration layer to have different evolution speeds.

Go remains the application backbone.

Python becomes the intelligence layer.

That distinction is more important than the language itself.

LangChain: From LLM Calls to Tools

The first implementation of the assistant was essentially an LLM call with conversation history.

That worked.

But it meant that my backend was responsible for finding the relevant context before calling the model.

The agent changed this.

I used LangChain's agent abstraction with LangGraph underneath it, allowing the model to decide whether to invoke a tool and continue reasoning after receiving the tool result.

The execution became:

The important part is that the application no longer hard-coded the complete sequence.

The user expresses an outcome.

The agent determines a possible execution path.

The First Real Agentic Test

I deliberately created a small set of tools with mocked CRM data.

The goal was not to build the entire CRM schema.

The goal was to test the agentic behavior.

For example:

Then I asked:

"Tell me everything important about my relationship with Salesforce. Look at our past discussions, check the current opportunity status, and tell me what I should follow up on next."

The agent independently performed a sequence similar to:

The important observation was not that three functions were called.

It was that the user never specified the workflow.

The user expressed the goal.

The agent selected the capabilities.

Tool Results Can Become Inputs to Other Tools

The next experiment was even more interesting.

The agent found an opportunity first:

It then used that result as the input to:

This produces a chain:

This is an important property of agentic systems.

Tools don't necessarily operate independently.

One tool can produce the information required to invoke another tool.

That allows the system to construct a workflow dynamically.

The Bigger Transition: Read → Reason → Write

Retrieval is useful.

But a CRM becomes much more interesting when the system can also act.

I added tools for operations such as:

Then I asked:

"Review my recent interactions with Salesforce and update the opportunity if you think anything important has changed. Also make a note explaining what you changed and why."

The agent produced a workflow similar to:

The user did not specify:

"Call update_opportunity."

The user expressed a business objective.

The agent translated the objective into system actions.

That is the point where an AI layer starts becoming more than a conversational interface.

Conversation Becomes Working Context

Another important transition was conversation state.

An agent does not automatically "remember everything" simply because it is an agent.

The system still needs to maintain the state required to continue a conversation.

In the current implementation, LangGraph's checkpointer maintains the state associated with a conversation thread.

The application can associate the conversation with a thread identifier:

This enables interactions such as:

"What are the biggest risks with Salesforce?"

followed by:

"That's useful. Make a note of it."

The second request does not contain the original information again.

The agent uses the conversation context to resolve what "it" refers to and can invoke the appropriate tool.

That was an important realization for me:

Conversation state is not merely a chatbot feature. In an agentic system, it becomes part of the execution context.

From Answering to Acting

This is where the architecture starts to resemble an assistant rather than a search interface.

Consider:

"That's a useful piece of information. Make a note of it."

The agent has to resolve several things:

The resulting tool call becomes something like:

The agent then receives the tool result and confirms the action.

This is a small example, but it represents a fundamental shift:

The system is beginning to offload administrative work rather than merely helping the user search for information.

Not Every Action Should Be Autonomous

The moment an agent can write data, another architectural question appears:

How much authority should the agent have?

Reading an email is one thing.

Updating an opportunity is another.

Sending an email to a customer is different again.

That led to human-in-the-loop behavior.

For external communication, I added a human approval boundary.

The execution looks like:

What I found particularly interesting was that the approval did not require me to build a completely separate conversational workflow.

The agent reached the approval boundary, asked:

"Can I go ahead and craft the email and send it?"

The user responded:

"Yes."

The same conversation continued and the tool execution proceeded.

The agent's execution state and the user's conversational state therefore meet at the approval boundary.

Drafting and Sending Are Different Capabilities

This experiment also exposed another important lesson.

There is a meaningful difference between:

and:

A draft has no external side effect.

Sending an email does.

This seems obvious from a product perspective, but it becomes an important architectural distinction when an LLM is choosing tools.

The action space should reflect the business semantics.

The more consequential the action, the stronger the control boundary should be.

Human-in-the-loop is therefore not simply a feature of the UI.

It is part of the authority model of the agent.

What I Learned About Agent Design

The most valuable part of this experiment has not been writing the tools.

It has been observing what happens when the system is given freedom to choose among them.

1. Agents do not reason exactly the way humans do

If I receive a request such as:

"Review the recent Salesforce interactions, understand the opportunity, and update it if necessary."

I may immediately imagine a specific workflow.

An LLM does not have to follow my imagined workflow.

It has access to the context, instructions, tools, and their descriptions and constructs its own path.

That is both the power and the risk.

Therefore, agent design is less about writing a perfect sequence and more about designing a bounded decision space.

2. The system prompt is part of the architecture

The system prompt is not merely a piece of text attached to the model.

It defines:

• the agent's responsibility,

• its boundaries,

• what it can assume,

• what it must not invent,

• how it should use tools,

• and when it should ask the user.

My current agent instructions explicitly require it to:

• work only with user-provided or tool-retrieved information,

• avoid hallucination,

• use tools for actions that change data,

• resolve conversational references,

• and never claim that an action succeeded unless the tool returned success.

These are architectural constraints expressed in natural language.

The prompt therefore becomes part of the system design.

3. Tool design matters as much as prompt design

A tool should expose a business capability, not simply expose a database table.

For example:

is more useful to an agent than:

The former gives the model a constrained action space.

The latter gives it a database.

That distinction becomes increasingly important as the number of tools grows.

I also found that CRUD operations often deserve separate tools with separate responsibilities.

Instead of one broad:

prefer:

The descriptions tell the agent when a capability should be used.

4. One tool should have one clear responsibility

This principle is deceptively simple.

A tool called:

could mean:

• search email,

• draft email,

• send email,

• reply to email,

• summarize email.

That creates ambiguity.

A better capability boundary is:

Each tool becomes easier for:

• the model to select,

• the developer to test,

• the system to secure,

• and the application to audit.

5. One agent does not necessarily mean one tool

I initially considered whether the system should immediately become multi-agent.

The experiments changed my view.

A single agent with five well-defined tools was able to:

• retrieve information,

• choose multiple tools,

• use one tool's output as another tool's input,

• reason across sources,

• update CRM data,

• create notes,

• maintain conversational context,

• and pause for human approval.

That is already a meaningful agent.

So I don't think "multi-agent" should be the starting point simply because the system is complex.

A better principle is:

Split agents when responsibilities, context, permissions, or reasoning loops become meaningfully different—not simply because the tool count increases.

A future architecture may eventually look like:

But that should emerge from a real complexity boundary.

6. Do not flood an agent with tools

More tools do not automatically mean a smarter agent.

Every additional tool adds:

• selection complexity,

• description complexity,

• testing combinations,

• permission considerations,

• failure modes,

• and more possibilities for the model to take an unnecessary path.

The objective is not to give the agent access to everything.

The objective is to give it the right capabilities for the responsibility it owns.

7. Data quality becomes even more important

RAG taught me that retrieving the right information matters.

Agentic systems made that lesson stronger.

The model only reasons over the information it receives.

If the knowledge base contains:

• raw HTML,

• duplicated content,

• signatures,

• irrelevant headers,

• tracking information,

• boilerplate,

• broken MIME extraction,

• and unrelated attachments,

the agent has to reason through all of it.

More data is not necessarily more intelligence.

For example, instead of embedding an entire HTML email, a system might extract:

and remove unnecessary representation before indexing.

Likewise, large content should be chunked appropriately rather than blindly embedded as one document.

In my current implementation, knowledge items are separated from embedding chunks, with chunking based on token limits and overlap before embeddings are generated.

The principle is simple:

Give the model high-quality information, not merely a large amount of information.

8. The backend should remain the authority

An agent should not become the source of truth.

The agent reasons.

The backend owns the system.

That distinction is critical.

The LLM should never be trusted to decide tenant boundaries, permissions, or whether a database mutation is technically valid.

The agent can request an action.

The application decides whether that action is allowed and executes it.

This keeps intelligence and authority separate.

The Architecture Is Becoming a Layer of Abstraction

This is perhaps the most important product and architecture insight from the experiment.

The agent layer sits between the user's intention and the application's capabilities.

The user does not need to understand the internal system.

They should be able to say:

"Review the Salesforce interactions and update the opportunity if something important has changed."

The intelligence layer translates that intent into system operations.

This is what I mean when I think about an AI-native product.

AI is not another screen placed on top of an existing database.

AI changes the interaction model of the product.

From CRM Interface to Intelligent Workspace

This connects back to the original product hypothesis.

A conventional CRM asks the user to maintain the system.

An intelligent CRM should increasingly maintain itself from the user's work.

The distinction is subtle but fundamental.

Traditional interaction

Agentic interaction

The second model does not eliminate the CRM.

It changes who carries the cognitive and administrative load.

What This Changes for Product Design

This experiment also changed how I think about AI features.

A weak AI feature looks like:

"We have a CRM and added an AI chatbot."

The database remains the center of the product.

The AI is an accessory.

A stronger model is:

"The system understands the user's intent and uses the CRM capabilities to accomplish the task."

The database becomes infrastructure.

The interaction becomes the product.

That is a much more interesting design space.

It also changes what a "feature" means.

A feature may no longer be a screen or a CRUD operation.

It could be:

"Prepare my next customer interaction."

Behind that one capability may be:

The user sees one capability.

The system orchestrates many operations.

Where the Current POC Ends

This is still a POC.

The current implementation deliberately keeps some pieces simple so that I can validate the architecture before investing in the complete CRM.

For example:

• some agent tools currently use mocked data,

• the Go event bus is in-memory,

• conversation checkpointing is currently in-memory,

• the CRM schema is not yet fully implemented,

• Gmail synchronization still needs production hardening,

• authorization boundaries around agent actions need to be enforced,

• and the tool layer still needs to evolve from local mocks to real backend APIs.

That is intentional.

The objective at this stage is not to claim that the entire CRM is production-ready.

The objective is to prove the interaction and orchestration model.

And that model is now working.

What I Would Build Next

The next phase is not "add more AI."

It is to identify the work that the system should take away from the user.

Some of the capabilities I am exploring are:

Relationship intelligence

"What have I discussed with Salesforce recently?"

Opportunity intelligence

"Which opportunities are at risk?"

Follow-up management

"What am I waiting for from customers?"

Communication

"Draft a follow-up based on our last conversation."

CRM maintenance

"Update the opportunity based on what we discussed."

Business preparation

"Prepare me for my conversation with Salesforce."

These capabilities can then be mapped to tools, agents, data sources, permissions, and workflows.

That is a more useful way to discover the architecture than deciding upfront that the system must be multi-agent.

The Road Ahead: From Agent to Agentic System

The current architecture has one agent and a small set of tools.

The next evolution may require explicit orchestration.

For example:

Whether this becomes necessary will depend on the number of capabilities, the complexity of the reasoning, the context each capability needs, and the permission model.

That is where LangGraph becomes increasingly interesting: explicit state, routing, branching, resumable execution, and human-in-the-loop workflows can become first-class parts of the architecture.

But I don't want to introduce complexity merely because a framework makes it possible.

The system should earn its complexity.

A Different Definition of "Intelligent"

The biggest lesson from this experiment is that intelligence in an application is not simply the quality of the model.

It is the combination of:

A powerful model with poor data can produce poor answers.

A good prompt with poorly designed tools can produce poor actions.

Good tools with unclear permissions can create unacceptable risk.

And a technically sophisticated agent with the wrong product objective can simply automate the wrong work.

The intelligence therefore lives in the system, not only in the model.

Final Reflection

I started this journey thinking about how to integrate Gmail into the system and make information available.

The experiment has moved much further.

The first milestone was:

Stop making the user search.

The next became:

Stop making the user assemble the answer.

The current question is:

Stop making the user perform the administrative work that follows from the answer.

That progression changes the role of the CRM.

It moves from:

toward:

I don't think the goal is to remove humans from the workflow.

The goal is to remove the work that humans should not have to spend their cognitive energy doing.

The best version of this CRM may therefore not feel like a CRM at all.

It may feel like having a capable assistant who knows the business context, remembers what matters, finds the information when needed, keeps the system current, prepares the next action, and asks for permission when the consequence is significant.

That is the direction I am now exploring.

From RAG to reasoning.
From retrieval to action.
From a CRM that stores the work to a system that helps do the work.

Contact

+91 - 9738482563

nagaraj.basarkod@yahoo.in