Retrieval Augmented Generation, commonly known as RAG, has become an important architecture for enterprise artificial intelligence applications. It allows large language models (LLMs) to work with company-specific information by retrieving relevant content from external data sources and supplying that information to the model when a response is generated. Instead of depending entirely on information learned during model training, a RAG application can retrieve current or proprietary information from enterprise databases, documents, knowledge bases, cloud storage, websites, and other approved sources.
RAG combines two major capabilities:
A simplified RAG workflow looks like this:
Enterprise data → document processing → embeddings → searchable index → user query → retrieval → relevant context → LLM → generated response
The retrieval component acts as a bridge between an AI model and information that may not have been included in the model's original training data.
This approach can be particularly useful for organizations with large collections of proprietary or frequently changing information.
General-purpose language models can generate fluent answers, but enterprise applications often need access to information that is private, specialized, or frequently updated.
Examples include:
RAG allows these sources to become searchable context for an AI application without requiring the language model to be retrained whenever a document changes.
AWS describes RAG as a practical enterprise approach for augmenting an LLM with external information such as internal company documents.
A production RAG system generally has two major phases: indexing and querying.
The indexing phase prepares enterprise information for retrieval.
A typical process is:
Microsoft describes ingestion as a key part of RAG, including content extraction, chunking, chunk organization, embedding, and update strategies.
When a user submits a question:
This separation allows enterprise content to be updated without retraining the underlying language model.
Embeddings are numerical representations of information.
An embedding model converts text or other content into a vector containing many numerical values. Content with related meanings can produce vectors that are relatively close together in vector space.
For example, documents discussing:
may be semantically related even if they do not use exactly the same words.
The retrieval system can use vector similarity to identify relevant content.
A vector database or vector-enabled search system stores embeddings and supports similarity searches.
A simplified record might contain:
| Field | Example |
|---|---|
| Document ID | HR-2045 |
| Text chunk | Employee leave policy... |
| Vector | Numerical embedding |
| Department | Human Resources |
| Region | North America |
| Access level | Internal |
| Updated | 2026 |
When a question arrives, the system can search for vectors that are similar to the question's embedding.
However, enterprise RAG systems do not always depend on vector search alone.
Keyword search remains valuable in enterprise environments.
Consider a question involving:
Product code AX-4729
A keyword search may be better suited to finding an exact identifier.
A semantic query such as:
"What procedure is used when a customer reports overheating?"
may benefit from vector search.
For this reason, enterprise RAG systems often combine keyword and vector retrieval.
Microsoft recommends evaluating vector, keyword, and hybrid approaches according to the data and query characteristics.
Hybrid search combines multiple retrieval methods.
A typical architecture can look like:
Keyword retrieval + vector retrieval → result fusion → reranking → top results
This can improve recall because semantic search and lexical search can identify different relevant documents.
Hybrid retrieval can be particularly useful for:
Azure AI Search documentation specifically recommends hybrid queries that combine keyword and vector search for improved recall in RAG scenarios.
Large documents generally need to be divided into smaller pieces before they are embedded.
This process is called chunking.
For example, a 100-page employee handbook might be divided into sections covering:
The goal is to create chunks that contain enough context to answer questions while remaining focused enough for accurate retrieval.
Large chunks may contain substantial irrelevant information.
This can:
Very small chunks can separate related information.
This may cause:
Microsoft's RAG architecture guidance treats chunking strategy as an important design decision that should be evaluated against the actual content and queries.
Metadata provides additional information about each chunk.
Useful metadata can include:
Metadata allows retrieval systems to narrow results.
For example:
Question: What is the warranty period?
Filter: Product = Model A
The system can then search only the relevant product documentation.
Filtering can improve both relevance and performance when used appropriately.
A RAG system can connect to many information sources.
Common examples include:
Microsoft's current RAG guidance highlights the challenge of bringing together enterprise content spread across multiple platforms while maintaining appropriate access controls.