Building an Arabic RAG Assistant That Answers Reliably

An Arabic RAG chatbot answers questions in Arabic from your own approved documents and cites the sources it used. Building one that answers reliably takes more than plugging Arabic text into a standard retrieval-augmented generation (RAG) pipeline: Arabic has many valid spellings of the same word, prefixes that attach to words, optional diacritics and widely used dialects, and many Arabic business documents are scans or bilingual PDFs. The reliable approach is to clean and normalize the text, retrieve with both keyword and meaning-based search, rerank the results, control the answer language, and test everything with real Arabic questions.

This guide covers each step, from document preparation to evaluation and deployment. If you are still choosing between a question-answering assistant and an agent that takes actions, start with our guide to RAG vs AI agents.

Why Arabic Makes RAG Harder

A RAG system succeeds or fails at retrieval: if the right passage is not found, even the best model answers from the wrong material. Arabic raises retrieval problems that English-first pipelines do not handle by default.

Rich morphology and attached prefixes

Arabic builds words from roots and patterns, and it attaches conjunctions, prepositions, the definite article and pronouns directly to words. A single written word such as “وبالعقد” (“and by the contract”) contains a conjunction, a preposition, the definite article and the noun. A keyword search for “عقد” will not match it unless the search engine splits or stems the word. One root also yields many related forms, so a question and the relevant policy may share meaning but no identical words.

Optional diacritics

Short vowels (diacritics, or tashkeel) are usually omitted in everyday writing. The unvocalized word “علم” can be read as “knowledge”, “flag” or “taught” depending on context. Most business documents are unvocalized, but some texts are partly vocalized, so the same word can appear in two forms across your documents.

Letter variants and inconsistent spelling

Arabic is not typed consistently. Common variations include:

• Alef and hamza forms: أ، إ، آ and ا are often used interchangeably, so “إدارة” may appear as “ادارة”.
• Taa marbuta and haa: a final “ة” is often typed as “ه”, so “مدرسة” may appear as “مدرسه”.
• Alef maqsura and yaa: “ى” and “ي” are frequently swapped at the end of words.
• Tatweel (kashida): the stretching character “ـ” used for visual justification, as in “مـــدير”.
• Digits: the same document set may mix Arabic-Indic digits (١٢٣) and Western digits (123).

To a computer, each variant is a different string, so a question typed one way misses a document written the other way.

Dialects and code-switching

Documents are usually written in Modern Standard Arabic (MSA), but people ask questions the way they speak. A Saudi employee may type “وش” or “ايش” instead of “ماذا” for “what”, or “ابغى” instead of “أريد” for “I want”. Many also switch between Arabic and English mid-sentence, such as “كيف أسوي approve للـ PO؟”, and some write Arabic in Latin letters with numbers (Arabizi). A reliable assistant must map all of these to the MSA and English terms used in your documents.

Preparing Arabic Documents: OCR, Extraction, Normalization and Chunking

Most wrong answers in Arabic RAG trace back to poor text, not a poor model.

Scanned documents and Arabic OCR

Older contracts, signed letters and stamped forms are often scanned images. Arabic optical character recognition (OCR) is harder than Latin OCR because the script is cursive, letters change shape by position, and dots are the only difference between letters such as ب، ت، ث and ن. Faded scans, stamps over text and low resolution lose those dots and change the word. Practical steps:

• Prefer the original digital file over a scan, and rescan important documents at good resolution.
• Choose an OCR engine by testing it on a sample of your own Arabic documents, including tables and handwriting if relevant.
• Flag pages with low OCR confidence for human review instead of indexing them silently.

Text extraction and direction

Even digital PDFs can yield broken Arabic on extraction: reversed characters, letters stored as Unicode “presentation forms”, or Arabic and English fragments out of order on mixed-direction lines. Check extracted text from every document type by eye. Unicode compatibility normalization (NFKC) converts presentation forms back to standard letters, but test its effect on your content before applying it everywhere.

Normalization choices

Normalization makes equivalent spellings identical. The rule that matters most: apply exactly the same normalization to indexed text and to user questions, and keep the original text for display and citations.

For keyword search, add light stemming or lemmatization so attached prefixes and suffixes do not block matches. Many search engines include an Arabic text analyzer for this; test it with your own terms, since aggressive stemming can merge words with different meanings.

Chunking right-to-left and bilingual documents

Chunking splits documents into retrievable passages. For Arabic content:

• Split on headings, articles and clauses, not a fixed character count, and keep the document title, section and page as metadata.
• For side-by-side bilingual documents, index each language as its own chunk sharing the same section reference.
• Convert tables into rows that repeat the column headers, so a retrieved row still makes sense on its own. Check the column order, since right-to-left tables are often extracted with columns reversed.

Retrieval: Multilingual Embeddings, Hybrid Search and Reranking

Multilingual embedding models

An embedding model turns text into numbers that represent meaning. Multilingual embedding models place Arabic and English text in the same space, so an Arabic question can find an English passage and the reverse. Embedding models differ considerably in Arabic quality and dialect coverage, so shortlist a few and compare them on a test set built from your own documents and questions rather than relying on general rankings.

Hybrid keyword and vector search

Vector search finds passages with similar meaning even when the wording differs, which helps with dialect and paraphrased questions. Keyword search finds exact terms that vector search can miss: contract numbers, item codes, names, clause references and specialist terms. Hybrid retrieval runs both and merges the results, for example with reciprocal rank fusion. For Arabic, this combination is usually more robust than either method alone.

Query rewriting

Before searching, a language model can rewrite the question: converting dialect into MSA, expanding abbreviations and adding English equivalents of key terms. Search with both the original and the rewritten question, and log both.

Reranking

Retrieval casts a wide net; a reranker then reads the question alongside each candidate passage and reorders them by relevance. Use a reranker that supports Arabic and send only the top passages to the model, which reduces noise and improves accuracy.

Controlling Answer Language and Citations

Write the language rules into the system instructions:

• Answer language. Usually the language of the question, in MSA even when the question is in dialect.
• Terminology. Keep official terms, product names and codes exactly as in the source rather than translating them.
• Dates and numbers. Saudi documents may use Hijri or Gregorian dates. Instruct the model to state which calendar a date uses and not to convert dates unless asked.
• Quoting across languages. When the answer is in Arabic but the source is English, show the original passage in the citation so the reader can check the translation.

Every answer should cite the document, section and page behind each point, with a link that opens the source. Instruct the assistant to say clearly when the approved sources do not contain the answer, rather than filling the gap from general knowledge.

Evaluating Answers in Arabic

A good demo proves little. Measure the assistant before launch and whenever the model, prompts, documents or settings change.

• Build an Arabic test set. Collect real questions from the people who will use the assistant, with correct answers and source passages confirmed by subject-matter experts. Include MSA and dialect phrasing, spelling variants, mixed Arabic-English questions and questions whose answer is not in the documents.
• Measure retrieval recall. For each question, check whether the correct passage appears among the retrieved results. If it does not, fix retrieval before touching prompts.
• Measure faithfulness. Check that every statement in the answer is supported by the cited passages, and that citations point to the right place.
• Check language quality. Confirm the expected language, sound grammar and correct terminology.
• Use native speakers. Automated scoring with a language model as judge helps at scale, but validate it against Arabic-speaking reviewers, who catch errors automated judges miss.

After launch, review real conversations regularly and add failures to the test set.

Access Control, PDPL and Deployment Choices

Permissions at retrieval. The assistant must never reveal a document the user could not open directly. Store access rights with each chunk and filter by the user’s identity before passages reach the model.

Personal data. Identify which documents contain personal data, exclude what the assistant does not need, and mask sensitive fields where possible. Decide how long questions, answers and retrieved passages are kept in logs, and who can read them. The Personal Data Protection Law (PDPL) and its regulations govern how personal data is processed, retained and transferred outside the Kingdom; confirm your specific obligations with your legal and compliance advisers.

Deployment and hosting. Every component that touches your data matters: language model, embedding model, reranker, vector index, logs and backups. For each, confirm where it is hosted, whether an in-Kingdom region is available or required for your data or contracts, whether prompts are retained or used for training, and whether a self-hosted model suits sensitive content better. Let your data classification drive the choice.

A Step-by-Step Build Plan

1. Pick one use case. A defined document set and audience, such as HR policies for employees.
2. Audit the documents. Find scans, bilingual files, tables and outdated versions; fix or remove poor sources.
3. Prepare the text. Run OCR where needed, verify extraction, define the normalization rules and chunk by document structure.
4. Build hybrid retrieval. Index with an Arabic-aware keyword analyzer and a multilingual embedding model, then add reranking.
5. Set language and citation rules. Write clear instructions on answer language, terminology, dates and declining when sources are silent.
6. Evaluate with an Arabic test set. Measure retrieval recall, faithfulness and language quality, with native-speaker review.
7. Secure and pilot. Apply permissions and logging, then pilot with a small group who report every wrong or unclear answer.
8. Monitor and expand. Track quality and cost, refresh the index as documents change, then add the next document set.

Common Mistakes

• Indexing text nobody has checked. Reversed characters, lost dots and scrambled tables quietly poison retrieval.
• Normalizing documents but not questions, or the reverse, so the two never match.
• Relying on vector search alone, which misses codes, names and exact clause references.
• Testing only in English or only in MSA, when real users ask in dialect and mixed language.
• Filtering permissions after generation, which can leak restricted content into an answer.

Frequently Asked Questions

Can one assistant handle both Arabic and English documents?

Yes. With a multilingual embedding model and hybrid retrieval, it can search both languages and answer in the user’s language. Test cross-language questions specifically.

Does the assistant understand Saudi dialect?

Modern language models generally understand common Gulf and Saudi expressions, but retrieval is the weak point because documents are written in MSA. Query rewriting into MSA and a test set that includes dialect questions address this.

Should we remove diacritics from our documents?

Remove them from the searchable copy so vocalized and plain text match, but keep the original text for display and citations. That gives you consistent search without altering what users read.

What if most of our documents are scanned?

Measure OCR quality on a representative sample first. Obtain digital originals or rescans for the most important documents, and route low-confidence pages to human review.

Do we need to train our own Arabic model?

Usually not. Careful document preparation, normalization, hybrid retrieval, reranking and evaluation solve most reliability problems with existing models. Fine-tuning can help in narrow cases but does not replace retrieval from current, citable sources.

AIMLTS, based in Jeddah, builds RAG assistants grounded in your documents and ERP data, with document AI for extraction and OCR, access controls, evaluation and guardrails designed in from the start. Learn more about our AI Agents & Automation service, or book a discovery call to discuss your Arabic document set and first use case.