Documentation

Learn how to implement our fine-tuning + RAG setup

Setup

Environment Setup

First, you'll need to set up your environment with the necessary dependencies. We recommend using a virtual environment.

bash
# Create a virtual environment
python -m venv .venv

# Activate the environment
# On Windows
.venv\Scripts\activate
# On macOS/Linux
source .venv/bin/activate

# Install dependencies
# On windows
pip install torch transformers datasets accelerate huggingface-cli unsloth
#On macOS/Linux
pip install torch transformers datasets accelerate huggingface-cli mlx-lm mlx

Data Preparation

Prepare your dataset for fine-tuning. The dataset should be in a format compatible with the Hugging Face datasets library.

python
import datasets

# Load your dataset
dataset = datasets.load_dataset("your_dataset_path")

# Preprocess the dataset
def preprocess_function(examples):
    return {
        "input_ids": tokenizer(examples["text"], truncation=True, padding="max_length"),
        "labels": examples["labels"]
    }

tokenized_dataset = dataset.map(preprocess_function, batched=True)

Retrieval System Setup

Set up the retrieval system that will be used during fine-tuning and inference.

python
from research_platform.retrieval import DocumentStore, Retriever

# Initialize document store
doc_store = DocumentStore(embedding_model="research-platform/embeddings")

# Add your documents
doc_store.add_documents("path/to/your/documents")

# Initialize retriever
retriever = Retriever(document_store=doc_store, top_k=5)

Training

Model Initialization

Initialize the base model that will be fine-tuned.

python
from research_platform import Model, ModelConfig

# Initialize configuration
config = ModelConfig.from_pretrained("research-platform/7b-base")
config.retrieval_augmentation = True

# Initialize model
model = Model.from_pretrained(
    "research-platform/7b-base",
    config=config
)

Fine-tuning Process

Run the fine-tuning process with retrieval augmentation.

python
from research_platform.trainer import RAGTrainer

# Initialize trainer
trainer = RAGTrainer(
    model=model,
    retriever=retriever,
    train_dataset=tokenized_dataset["train"],
    eval_dataset=tokenized_dataset["validation"],
    args={
        "output_dir": "./results",
        "num_train_epochs": 3,
        "per_device_train_batch_size": 4,
        "gradient_accumulation_steps": 8,
        "learning_rate": 2e-5,
        "warmup_steps": 500,
    }
)

# Start training
trainer.train()

Co-training Loop

Implement the co-training loop to iteratively improve both the model and retrieval system.

python
from research_platform.cotraining import CoTrainingLoop

# Initialize co-training loop
co_training = CoTrainingLoop(
    model=model,
    retriever=retriever,
    dataset=dataset,
    num_iterations=5
)

# Run co-training
co_training.run()

Inference

Loading Fine-tuned Model

Load your fine-tuned model for inference.

python
from research_platform import Model

# Load fine-tuned model
model = Model.from_pretrained("./results")

Inference with Retrieval

Run inference with retrieval augmentation.

python
# Generate text with retrieval augmentation
response = model.generate(
    "Explain the advantages of fine-tuning with retrieval augmentation",
    use_retrieval=True,
    max_length=512,
    temperature=0.7
)

print(response)

Chain-of-Thought Reasoning

Utilize chain-of-thought reasoning for complex problems.

python
# Generate text with chain-of-thought reasoning
response = model.generate(
    "Solve this math problem step by step: If a train travels at 60 mph for 2 hours and then at 80 mph for 1 hour, what is the average speed?",
    use_retrieval=True,
    chain_of_thought=True,
    max_length=1024
)

print(response)