From Emails to Conversations: Engineering a RAG Pipeline

Computing has always been evolving around one fundamental question: how should a human interact with a machine?

In the early days, the interaction was explicit. We instructed computers what to do. As computing power and storage grew, systems became capable of holding and processing increasingly large amounts of information. The interaction gradually shifted from instructing the computer to finding information from it.

Search changed that experience.

We stopped having to know where information was stored. We could describe what we were looking for, and systems such as Google and later enterprise search platforms could help us find it.

The next shift is not simply better search.

It is a different interaction model.

Instead of asking the system to help us find information, we increasingly expect the system to understand what we are trying to accomplish and help us get there. We ask questions in natural language. We expect context to be understood. We expect the system to combine information from different sources and return something useful rather than a list of documents.

That shift from search to conversation looks like a user experience change on the surface.

Underneath, it changes what we expect the system to do with the information it already has. The integration pipeline still has to collect and store information, but now that information also needs to become usable knowledge. It needs to be indexed semantically, retrieved by intent, and assembled into context that an AI model can reason over.

From an email to an answer

At a high level, the system follows a fairly simple path.

An email enters the system through Gmail.

The integration authenticates with Gmail, synchronizes new messages and hands them to the email layer. The email layer stores the normalized representation and publishes an event.

That event starts the knowledge pipeline.

The knowledge service turns the email into a knowledge item and publishes another event. The embedding pipeline consumes that event, chunks the content when necessary, generates embeddings and stores them alongside the chunks.

When a user asks a question, the query itself goes through the same semantic representation. The system retrieves relevant knowledge, assembles the retrieved context into an augmented prompt and sends that context to the language model.

The model generates the response.

The complete flow is therefore:
External Information
↓
Integration
↓
Normalized Domain Data
↓
Knowledge
↓
Chunking
↓
Embeddings
↓
Semantic Retrieval
↓
Context Assembly
↓
LLM
↓
Conversation

The LLM is only one step in that pipeline.

The quality of the final interaction depends heavily on everything that happens before it.

The idea

I wanted to explore what it would take to bring this interaction model to the information we already generate every day.

Email was a natural starting point.

Email contains conversations, decisions, commitments, documents, relationships and a large amount of contextual information. Yet we still interact with it largely through folders, filters and search boxes.

What if I could simply ask:

"Do I have any emails that talk about the wine industry?"
Or:
"What did I discuss with this customer last month?"
Or:
"Did anyone send me something related to the opportunity?"

The interesting part is not generating the answer.

The interesting part is building the system underneath that can reliably turn distributed information into the context required to answer the question.

That became the engineering exercise.

The objective was to build a baseline that could connect an external information source, continuously ingest its data, transform it into knowledge, make that knowledge searchable through semantic retrieval, and finally use retrieved context to respond to a natural language query.

Gmail was the first integration. It is not the destination.

The architecture is intended to become a foundation for additional integrations and future AI capabilities.

Why the architecture matters

A RAG implementation can look deceptively small.

Take some text. Generate an embedding. Put it in a vector database. Search for similar vectors. Add the results to a prompt. Ask an LLM for an answer.

The code for those individual operations is not particularly complicated.

The engineering becomes more interesting when the system has to operate as a real application.

• Where does provider-specific logic live?

• Who owns the email?

• Who decides what becomes knowledge?

• How does a worker know what to process?

• What happens when an email is too large for the embedding model?

• How are multiple integrations going to coexist?

• Where does semantic search belong?

• What happens when the source data changes?

• How do we keep the language model from becoming responsible for things that should be deterministic?

These questions led to a deliberately separated architecture.

The goal was not to create as many services as possible. It was to establish meaningful boundaries around responsibilities.

Each component should know what it owns, and should not need to know how another component implements its responsibility.

That distinction is important.

Modularity is not about having many small classes or packages. It is about creating boundaries that remain meaningful as the system grows.

The integration boundary

Gmail is a provider.

The rest of the system should not need to understand Gmail's internal representation of an email.

The Gmail integration handles OAuth2, token management, synchronization, Gmail history, message retrieval and Gmail-specific message structures.

It converts those provider-specific structures into a normalized email representation.

This means that the rest of the pipeline can work with an email rather than a Gmail message.

That distinction becomes increasingly important when more integrations are introduced.

Gmail may represent a message one way. Microsoft Graph may represent it differently. An IMAP integration may expose a raw MIME message.

Those differences belong at the edge.

The internal pipeline should not care.

This also influenced the decision to keep the email service intentionally simple. It acts as the bridge between the integration and the application's email data. It stores the normalized email and emits the event that tells the rest of the system that something new exists.

The integration does not need to know what the knowledge pipeline will eventually do with that email.

Events create the next boundary

Once an email is stored, the system publishes an EmailStored event.

The knowledge worker listens for that event.

It does not need to know that Gmail produced the email. It only knows that an email now exists and needs to become knowledge.

The knowledge service retrieves the email, creates a normalized KnowledgeItem, stores it and publishes a KnowledgeItemCreated event.

The embedding worker listens to that event.

It does not need to know how the knowledge item was created.

It simply passes the knowledge item into the embedding service, which takes care of chunking, generating embeddings and updating the vector index.

The pipeline therefore becomes asynchronous:
Email
↓
EmailStored
↓
KnowledgeWorker
↓
KnowledgeItem
↓
KnowledgeItemCreated
↓
EmbeddingWorker
↓
Chunks + Embeddings

This separation gives each stage a clear responsibility while allowing the pipeline to evolve independently.

If another source begins producing knowledge items tomorrow, the embedding pipeline does not need to know where they came from.

If the embedding strategy changes, the Gmail integration does not need to change.

That is the kind of scalability I was looking for.

Not simply being able to process more records, but being able to introduce more capabilities without forcing unrelated components to change.

Knowledge is not the same thing as an embedding

One of the important design decisions was separating the knowledge model from its semantic index.

An email is a business object.

A knowledge item represents that object in the knowledge layer.

An embedding is an indexing representation of that knowledge.

They are related, but they are not the same thing.

This became especially clear when the first large email exceeded the embedding model's input limit.

The initial mental model was simple:
KnowledgeItem
|Content
|Embedding

That works while one knowledge item produces one embedding.

It stops being a good model when one large item needs to be split into multiple chunks.

The better representation is:
KnowledgeItem
|Chunk 1 → Embedding
|Chunk 2 → Embedding
|Chunk 3 → Embedding
|…

The knowledge item remains the canonical representation.

The chunks and their vectors become an indexing structure used for retrieval.

This distinction also leaves room for future re-indexing. The underlying knowledge does not have to change simply because the embedding model or chunking strategy changes.

Chunking is an engineering concern, not an LLM feature

The first large email made another point very clear.

Embedding models operate within token limits.

A piece of content that looks reasonable in characters can still be too large in tokens.

The embedding pipeline therefore needs to understand the size of the content before sending it to the model.

For the current implementation, token counting is handled with tiktoken, while text splitting can be delegated to a text-splitting library such as LangChainGo.

The important architectural point is that neither the worker nor the knowledge service needs to know how chunking works.

The embedding service owns that concern.

That means the system can later move from character-based splitting to token-aware splitting, or eventually to more sophisticated semantic chunking, without changing the rest of the ingestion pipeline.

The exact chunking strategy is an implementation choice.

The boundary is the important part.

Why 'pgvector'

Vector storage was another deliberate choice.

There are many specialized vector databases and hosted vector services. They are useful options, particularly at larger scales or when a system needs capabilities specific to a dedicated vector platform.

For this baseline, I chose PostgreSQL with pgvector.

The decision was pragmatic.

The system already needs relational data. The amount of data being handled initially does not justify introducing another operational dependency purely for vector storage. PostgreSQL gives me a mature transactional database, familiar querying and indexing, and pgvector provides the vector similarity capabilities required for semantic retrieval.

It also keeps the architecture simple.

That matters.

A good architecture is not the one with the most specialized components. It is the one that introduces the right components when they solve a real problem.

As the scale and requirements change, the storage decision can change with them.

Why Go

The implementation is written in Go.

This was not a decision to prove that Go is better than Python, Java or .NET. Those ecosystems have excellent use cases and mature production deployments.

The decision was more personal and practical.

I have spent a significant part of my career building production software in Swift, and Go felt like a natural transition into backend systems. There are some similarities in the language philosophy and the emphasis on keeping the language relatively small and understandable.

Go also fits the nature of the system well.

There is a lot of integration work, HTTP communication, database interaction, concurrency and event processing. The AI model is an external capability being invoked by the system, not the system itself.

The language therefore does not need to be an AI research environment.

It needs to be good at building the system around the AI.

That distinction helped keep the architecture grounded in backend engineering rather than making every part of the system an AI framework.

The query path is intentionally different from the ingestion path

The ingestion pipeline prepares knowledge.

The query pipeline consumes it.

A user query first needs to be represented in the same semantic space as the stored knowledge. The query is therefore converted into an embedding and used to perform semantic retrieval against the indexed knowledge chunks.

The retrieved chunks then become context.

That context is assembled into an augmented prompt and passed to the language model.

The model is responsible for generating the response from the supplied context.

The simplified query flow is:
User Question
↓
Query Embedding
↓
Vector Search
↓
Relevant Knowledge Chunks
↓
Context Assembly
↓
Augmented Prompt
↓
LLM
↓
Response

This separation is important because search and generation solve different problems.

Search determines what information is relevant.

RAG determines how that information is presented to the model.

The model determines how to formulate the response.

Keeping those responsibilities distinct makes the system easier to reason about and gives each part room to evolve.

The LLM is not the system

This is perhaps the most important lesson from the exercise.

It is tempting to look at an AI application and see the LLM as the centre of everything.

In practice, the LLM is one component in a larger system.

It does not synchronize Gmail.
It does not maintain the knowledge index.
It does not decide which records are available.
It does not guarantee that the retrieved information is correct.
It does not automatically remember previous requests.

The surrounding system has to provide those capabilities.

Even conversation continuity is an application responsibility. A language model responds to the context supplied to it. If the application does not preserve and provide the relevant conversation state, that continuity does not magically exist.

That becomes even more important as the system evolves from simple RAG into tool-using agents.

Designing for the next capability

The current system deliberately stops at the RAG boundary.

That is not because the architecture ends there.

It is because the next problem is different.

Today the system can answer questions from indexed knowledge:
Question
↓
Retrieve
↓
Augment
↓
Respond

The next step is to allow the system to take actions.

That introduces tools.

For example, instead of only retrieving email content, a future system could expose capabilities such as:
Search Email
Read Email
Search Contacts
Read Calendar
Create Task
Update Opportunity

The language model can then become responsible for deciding which capabilities are required to satisfy an intent.

The architecture evolves from:
RAG

towards:

Reasoning
↓
Tool Selection
↓
Tool Execution
↓
Observation
↓
Further Reasoning
↓
Response

The important point is that the foundation does not need to be thrown away.

The integration boundaries, knowledge layer, event pipeline, search layer and data model remain useful.

Agentic behavior becomes another layer over an existing system rather than a replacement for it.

What this exercise demonstrated

The interesting outcome for me is not that Gmail can now be queried using an LLM.

That is a useful capability, but it is not the difficult part.

The more important result is the baseline architecture.

An external information source can enter through an integration boundary. The information can be normalized, persisted and transformed into knowledge. Events can move that knowledge through asynchronous processing stages. The knowledge can be chunked and semantically indexed. A query can retrieve relevant context and provide that context to a language model.

The system can therefore move from:

External Information

to:

Conversational Context

without making any individual component responsible for the entire journey.

That is the architectural capability I wanted to establish.

From emails to a broader information layer

Gmail is only the first integration.

The same architecture can be extended to other information sources.

A different integration can synchronize its data and normalize it into the appropriate domain representation. The downstream knowledge and embedding pipeline can continue to operate without needing to understand the provider.

That opens the door to a broader information layer.
Email.
Documents.
Calendar.
Notes.
Other communication systems.

Eventually, the system can become a place where information from multiple parts of a person's digital life is available through one interaction model.

Instead of visiting each system independently to understand what is happening, the user can ask.

That is the product idea behind the technical exercise.

And that is why I started with the interaction model rather than the technology.

The technology makes the interaction possible.

The architecture makes it sustainable.

Contact

+91 - 9738482563

nagaraj.basarkod@yahoo.in