Fine-Tune with Ollama

Create custom Ollama models by embedding your training data in the system prompt. No actual training required - perfect for quick testing!

📋 Prerequisites

📦

1. Dataset Ready

Export your dataset in any format from the EdukaAI application Export page. Ollama works with the raw JSON.

Open the EdukaAI app and go to Export.

💻

2. Ollama Installed

Ollama must be installed and running on your machine.

Download from ollama.com →

💡 How This Works

Unlike other methods that train model weights, Ollama uses a Modelfile to embed your training examples directly in the system prompt. The model learns patterns from your examples without any training time!

✅ Pros

• Instant - no waiting
• No GPU required
• Works on any computer
• Easy to update data

⚠️ Cons

• Limited by context size
• Not true fine-tuning
• Best with small datasets

🎯 Best For

• Quick testing
• Prototyping
• 5-50 examples
• Pattern-based tasks

1 Install Ollama

Quick install (macOS/Linux):

curl -fsSL https://ollama.com/install.sh | sh

🍎 macOS

Download .dmg from ollama.com

🪟 Windows

Download installer from ollama.com

Verify installation:

ollama --version

2 Pull a Base Model

Choose a small, fast model. Your training data will be embedded in its system prompt.

llama3.2:1b ⭐

Fastest, smallest

ollama pull llama3.2:1b

phi3:mini

Good quality, fast

ollama pull phi3:mini

mistral:7b

Higher quality

ollama pull mistral:7b

💡 Tip: Smaller models (1B-3B) work better with this method because they have more room in their context window for your training data.

3 Create a Modelfile

The Modelfile embeds your training data into the model's system prompt. Create this file in the same folder as your exported JSON.

# Save as 'Modelfile' (no extension)

FROM llama3.2:1b

SYSTEM """You are a helpful assistant trained on specific examples. Use the following training data to guide your responses:

[PASTE YOUR EXPORTED JSON CONTENT HERE]

Answer questions based on patterns from the training examples above. Be helpful, accurate, and follow the style shown in the examples."""

PARAMETER temperature 0.7
PARAMETER top_p 0.9

📋 How to Paste Your Data

1. Open your exported JSON file
2. Select all content (Ctrl+A)
3. Copy (Ctrl+C)
4. Open your Modelfile
5. Replace [PASTE YOUR EXPORTED JSON CONTENT HERE] with the copied content

⚠️ Size Warning: Keep your training data under ~4000 tokens (roughly 3000 words). If your data is too large, the model may ignore parts of it.

4 Create Your Custom Model

Run this command in the same folder as your Modelfile:

ollama create my-custom-model -f Modelfile

✅ What Happens

• Ollama reads your Modelfile
• Embeds your training data as system instructions
• Creates a new model named "my-custom-model"
• Ready to use instantly!

5 Test Your Model

Interactive chat:

ollama run my-custom-model

Single prompt:

ollama run my-custom-model "How do I reverse a string in Python?"

🧪 Test Prompts

Try questions similar to your training data:

• "How do I reverse a string in Python?"
• "Explain photosynthesis simply"
• "What's the difference between let and const?"
• "Write a function to check if a number is prime"

🔍 Check if it's Working

If the model responds with patterns from your training examples (similar explanations, code style, or terminology), it's working!

6 Use Your Model in Applications

Ollama provides an OpenAI-compatible API. Your model is available at http://localhost:11434

# Python example

import requests

response = requests.post('http://localhost:11434/api/generate', json={
    'model': 'my-custom-model',
    'prompt': 'How do I reverse a string in Python?',
    'stream': False
})

print(response.json()['response'])

# JavaScript/TypeScript example

const response = await fetch('http://localhost:11434/api/generate', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'my-custom-model',
    prompt: 'How do I reverse a string in Python?',
    stream: false
  })
});

const data = await response.json();
console.log(data.response);

🌐 OpenAI Compatibility

Ollama also supports the OpenAI API format for easy integration:

 curl http://localhost:11434/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model": "my-custom-model", "messages": [{"role": "user", "content": "Hello!"}]}'

💡 Pro Tips

📏 Keep Data Small

Under 50 examples (or ~4000 tokens) for best results

🔄 Easy Updates

Just edit Modelfile and run ollama create again

🎯 Use Similar Examples

Group related examples together in your data

📚 Use Small Models

1B-3B models work better than 7B+ for this method

⚖️ Ollama vs True Fine-Tuning

Aspect	Ollama Method	Axolotl/LoRA
Training Time	Instant	Minutes to hours
Data Size Limit	~4K tokens	Unlimited
GPU Required	No	Recommended
True Learning	No (context only)	Yes (weights update)
Best For	Quick testing, prototyping	Production, large datasets

← All Methods