📪RAG

This page details the components of our RAG pipeline.

Key Components

1. Document Indexing: Uses llama-index and ragatouille libraries to process, chunk and index documents for efficient retrieval.

2. Abbreviation Handling: Incorporates abbreviation expansions into the prompt to enrich the context.

3. Answer Generation: Uses our fine-tuned model to generate answers based on a custom prompt.

4. Answer Parsing: Extracts and formats the model's response into a usable answer format.

Implementation Details

---

Document Indexing

The script uses a combination of llama-index and ragatouille to process and index documents. This process involves:

1. Loading the word documents using llama-index's SimpleDirectoryReader.

2. Splitting documents into chunks of 150 tokens.

3. Vectorizing chunks using ColBERT embeddings through ragatouille.

4. Creating an index of vectorized chunks which uses FAISS for efficient retrieval.

documents = SimpleDirectoryReader("data/rel18").load_data()

docs_str = []
for doc in documents:
    docs_str.append(doc.text)

RAG = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0")

RAG.index(
    collection=docs_str,
    index_name="ITU RAG 150",
    max_document_length=150,
    split_documents=True,
)

Abbreviation Handling

As mentioned in Abbreviations we use the abbreviation glossary to expand the prompt.

Answer Generation

Context Retrieval

For each question, we retrieve relevant context using the created index. This process fetches the top 13(Phi-2) and top 3(Falcon-7B) most relevant chunks from out indexed documents, which are then concatenated to form the context for the question at hand.

# Phi-2
results = RAG.search(query=question_text, k=13)
context = " ".join([result["content"] for result in results])

# Falcon-7B
results = RAG.search(query=question_text, k=3)
context = " ".join([result['content'] for result in results])

Prompt Creation

The contents of the prompt vary depending on the model. For Phi-2 the prompt includes:

1. An instruction for the model

2. The retrieved context

3. Relevant abbreviations and their expansions

4. The question

5. The answer options

6. A prefix for the answer prompt

def create_prompt(question, options, context, abbreviations):
    options_text = "\n".join(
        [f"Option {i+1}: {opt[1]}" for i, opt in enumerate(options)]
    )
    abbreviations_text = "\n".join(
        [
            f"{list(abbrev.keys())[0]}: {list(abbrev.values())[0]}"
            for abbrev in abbreviations
        ]
    )

    prompt = (
        f"Instruct: You will answer each question correctly using the context below\n"
        f"Context: {context}\n\n"
        f"Abbreviations:\n{abbreviations_text}\n\n"
        f"Question: {question}\n"
        f"{options_text}\n"
        f"Answer: Option"
    )
    return prompt

The structure of the prompt for Falcon-7B is described here Response Scoring (Falcon7B).