Documentation
Learn how to implement our fine-tuning + RAG setup
Setup
Environment Setup
First, you'll need to set up your environment with the necessary dependencies. We recommend using a virtual environment.
bash
# Create a virtual environment
python -m venv .venv
# Activate the environment
# On Windows
.venv\Scripts\activate
# On macOS/Linux
source .venv/bin/activate
# Install dependencies
# On windows
pip install torch transformers datasets accelerate huggingface-cli unsloth
#On macOS/Linux
pip install torch transformers datasets accelerate huggingface-cli mlx-lm mlxData Preparation
Prepare your dataset for fine-tuning. The dataset should be in a format compatible with the Hugging Face datasets library.
python
import datasets
# Load your dataset
dataset = datasets.load_dataset("your_dataset_path")
# Preprocess the dataset
def preprocess_function(examples):
return {
"input_ids": tokenizer(examples["text"], truncation=True, padding="max_length"),
"labels": examples["labels"]
}
tokenized_dataset = dataset.map(preprocess_function, batched=True)Retrieval System Setup
Set up the retrieval system that will be used during fine-tuning and inference.
python
from research_platform.retrieval import DocumentStore, Retriever
# Initialize document store
doc_store = DocumentStore(embedding_model="research-platform/embeddings")
# Add your documents
doc_store.add_documents("path/to/your/documents")
# Initialize retriever
retriever = Retriever(document_store=doc_store, top_k=5)Training
Model Initialization
Initialize the base model that will be fine-tuned.
python
from research_platform import Model, ModelConfig
# Initialize configuration
config = ModelConfig.from_pretrained("research-platform/7b-base")
config.retrieval_augmentation = True
# Initialize model
model = Model.from_pretrained(
"research-platform/7b-base",
config=config
)Fine-tuning Process
Run the fine-tuning process with retrieval augmentation.
python
from research_platform.trainer import RAGTrainer
# Initialize trainer
trainer = RAGTrainer(
model=model,
retriever=retriever,
train_dataset=tokenized_dataset["train"],
eval_dataset=tokenized_dataset["validation"],
args={
"output_dir": "./results",
"num_train_epochs": 3,
"per_device_train_batch_size": 4,
"gradient_accumulation_steps": 8,
"learning_rate": 2e-5,
"warmup_steps": 500,
}
)
# Start training
trainer.train()Co-training Loop
Implement the co-training loop to iteratively improve both the model and retrieval system.
python
from research_platform.cotraining import CoTrainingLoop
# Initialize co-training loop
co_training = CoTrainingLoop(
model=model,
retriever=retriever,
dataset=dataset,
num_iterations=5
)
# Run co-training
co_training.run()Inference
Loading Fine-tuned Model
Load your fine-tuned model for inference.
python
from research_platform import Model
# Load fine-tuned model
model = Model.from_pretrained("./results")Inference with Retrieval
Run inference with retrieval augmentation.
python
# Generate text with retrieval augmentation
response = model.generate(
"Explain the advantages of fine-tuning with retrieval augmentation",
use_retrieval=True,
max_length=512,
temperature=0.7
)
print(response)Chain-of-Thought Reasoning
Utilize chain-of-thought reasoning for complex problems.
python
# Generate text with chain-of-thought reasoning
response = model.generate(
"Solve this math problem step by step: If a train travels at 60 mph for 2 hours and then at 80 mph for 1 hour, what is the average speed?",
use_retrieval=True,
chain_of_thought=True,
max_length=1024
)
print(response)